Senior Data Engineer
Vimal KumarUnited StatesSENIORPosted 6d ago
Data bricksPySparkSQLStructured StreamingDelta Live TablesUnity CatalogTerraformData bricks Asset Bundles
About the Role
We are seeking a Senior Data Engineer / Data bricks Specialist to join our team. This role focuses on designing scalable data solutions and mentoring engineers.
Key Responsibilities
• Lead the architecture of data solutions leveraging Data bricks, including Delta Lake, Unity Catalog, and Medallion Architecture principles
• Own end-to-end data engineering initiatives, providing technical direction and guidance to the development team
• Design, develop, and review data pipelines to ensure scalability, maintainability, and engineering best practices
• Improve workload efficiency and implement strategic measures to manage and reduce compute costs
• Maintain robust standards for CI/CD, automated testing, monitoring, observability, and data governance
• Partner with product owners and stakeholders to translate business needs into technical solutions while mentoring junior engineers
Requirements
• 8+ years in Data Engineering + 5+ years of hands-on production experience delivering Data bricks solutions
• Strong expertise in PySpark and SQL
• Real-time data solutions using Structured Streaming and/or Delta Live Tables
• Hands-on experience with Unity Catalog, Terraform and/or Data bricks Asset Bundles, and at least one major cloud platform
• Proven experience leading, mentoring, or guiding engineering teams within an Agile/Scrum framework
• Preferred Data bricks Professional Certification