Senior Data Engineer, Ingestion Framework - Data & AI Platform at Rivianvw.tech

ABOUT US Rivian and Volkswagen Group Technologies is a joint venture between two industry leaders with a clear vision for automotive’s next chapter. From operating systems to zonal controllers to cloud and connectivity solutions, we’re addressing the challenges of electric vehicles through technology that will set the standards for software-defined vehicles around the world. The road to the future is uncharted. By combining our expertise across connectivity, AI, security and more, we’ll map a new way forward. Working together, we’ll create a future that’s more connected, more intelligent, more sustainable for everyone. Rivian and Volkswagen Group Technologies Canada is proud to be a Great Place To Work® Certified company — 92% of employees at RV Tech Canada say it is a great place to work, compared to 60% at a typical company. ROLE SUMMARY As a Senior Data Engineer in the Data & AI Platform team, you will play a pivotal role in enabling our big data platform to operate seamlessly at a petabyte scale. The successful candidate will be an architect and custodian of a custom-built, Go-powered vehicle data processing framework, ensuring its smooth integration with our Databricks platform. RESPONSIBILITIES - Framework Development & Maintenance: Design, develop, and enhance a cutting-edge, petabyte-scale vehicle data processing framework using Go. This framework is the backbone of our data operations; therefore, the ability to write clean, efficient, and maintainable code is essential. - Databricks Integration: Collaborate closely with the big data platform team to ensure seamless integration of the framework with our Databricks environment. This involves understanding the intricacies of both systems and optimizing their interaction for maximum efficiency. - CI/CD & Kubernetes Management: Take ownership of the framework's deployment lifecycle. Implement and manage robust CI/CD pipelines on Kubernetes to ensure smooth, automated, and reliable deployments. - Performance Optimization: Continuously monitor the framework's performance, identify bottlenecks, and implement optimizations to ensure it operates at peak efficiency even as data volumes grow. - Troubleshooting & Issue Resolution: Proactively identify and resolve issues that may arise with the framework. The ability to diagnose problems quickly and implement effective solutions will be critical to maintaining system uptime. - Collaborative Development: Collaborate with data producers and consumers to deliver dependable and scalable solutions. - Staying Ahead of the Curve: Keep abreast of the latest advancements in big data technologies, data engineering practices, and cloud computing. Explore new tools and techniques to improve our data infrastructure. QUALIFICATIONS - Education: Bachelor's or Master's degree in Computer Science, Data Science, or a related field. - Experience: 5+ years of hands-on experience in data engineering or a similar role, with a proven track record of building and maintaining large-scale data processing systems. - Go Expertise: Deep proficiency in Go programming language, with a strong understanding of its concurrency model and best practices for building high-performance applications. - Big Data Technologies: Solid understanding of big data ecosystems, such as Spark and Databricks. Experience with data warehousing and data lake concepts is a plus. - Problem-Solving Skills: Exceptional analytical and problem-solving skills, with the ability to break down complex problems into manageable components and develop effective solutions. - Communication & Collaboration: Excellent communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams. Nice to Have: - Kubernetes: Experience with Kubernetes container orchestration, deployment strategies, and cluster management. - Delta Lake: Familiarity interacting with open source or Databricks proprietary Delta tables. Understanding of Delta protocol is an extra bonus. TOTAL REWARDS Total compensation packages for full-time positions include base salary, eligibility for an annual performance bonus, and eligibility for equity. In addition, our benefits package has been designed to support the health and wellness of our employees. Benefit offerings include Flex Time Off, retirement savings plans as well as medical, vision and dental coverage. For more information on RV Tech’s comprehensive benefits package for full-time employees, check out our Global Benefits Site [Upgrade to PRO to see link] External candidates can apply for this role through the RV Tech Careers site ([Upgrade to PRO to see link] If you are a current employee, please apply through our internal job board [Upgrade to PRO to see link] EQUAL OPPORTUNITY Rivian and Volkswagen Group Technologies is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, ancestry, sex, sexual orientation, gender, gender expression, gender identity, genetic information or characteristics, physical or mental disability, marital/domestic partner status, age, military/veteran status, medical condition, or any other characteristic protected by law. We are also committed to ensuring compliance with all applicable fair employment practice laws regarding citizenship and immigration status. Rivian and Volkswagen Group Technologies is committed to ensuring that our hiring process is accessible for persons with disabilities. If you have a disability or limitation, such as those covered by the Americans with Disabilities Act, that requires accommodations to assist you in the search and application process, please email us at [Upgrade to PRO to see contact]. CANDIDATE DATA PRIVACY Rivian and Volkswagen Group Technologies” may collect, use and disclose your personal information or personal data (within the meaning of the applicable data protection laws) when you apply for employment and/or participate in our recruitment processes (“Candidate Personal Data”). This data includes contact, demographic, communications, educational, professional, employment, social media/website, network/device, recruiting system usage/interaction, security and preference information. Rivian and Volkswagen Group Technologies may use your Candidate Personal Data for the purposes of (i) tracking interactions with our recruiting system; (ii) carrying out, analyzing and improving our application and recruitment process, including assessing you and your application and conducting employment, background and reference checks; (iii) establishing an employment relationship or entering into an employment contract with you; (iv) complying with our legal, regulatory and corporate governance obligations; (v) record keeping; (vi) ensuring network and information security and preventing fraud; and (vii) as otherwise required or permitted by applicable law. Rivian and Volkswagen Group Technologies may share your Candidate Personal Data with (i) internal personnel who have a need to know such information in order to perform their duties, including individuals on our People Team, Finance, Legal, and the team(s) with the position(s) for which you are applying; (ii) Rivian and Volkswagen Group Technologies affiliates; and (iii) Rivian and Volkswagen Group Technologies’ service providers, including providers of background checks, staffing services, and cloud services. Rivian and Volkswagen Group Technologies may transfer or store internationally your Candidate Personal Data, including to or in the United States, Canada, and the European Union and in the cloud, and this data may be subject to the laws and accessible to the courts, law enforcement and national security authorities of such jurisdictions. If you provide a mobile telephone number as part of your application or during the recruitment process, Rivian and Volkswagen Group Technologies may use that number to contact you via SMS text message for recruitment-related purposes, including scheduling, logistics, and status updates. Message and data rates may apply. You may opt out of SMS communications at any time by replying STOP to any text message you receive from us. Consent to receive SMS messages is not a condition of applying for or being considered for employment. Please see our Candidate Data Privacy Notice (English) [Upgrade to PRO to see link] and Candidate Data Privacy Notice (Serbian) [Upgrade to PRO to see link] for more information. -- Please note this job posting represents an open, active vacancy. Additionally, we are not currently accepting applications from third party application services.