Data Engineer
Manish Kumar YadavUnited StatesSENIORPosted 6d ago
DatabricksDelta LakeUnity CatalogPySparkSQLMedallion ArchitectureStructured StreamingDelta Live TablesTerraformDatabricks Asset BundlesCloud PlatformsCI/CD
About the Role
We are looking for an experienced Data Engineer with strong hands-on Databricks expertise, data pipeline development, platform optimization, and technical leadership skills.
Key Responsibilities
• Develop and maintain data pipelines using Databricks and PySpark
• Implement Delta Lake and Unity Catalog for data management
• Design and optimize data architectures using Medallion Architecture
• Work with Structured Streaming and Delta Live Tables for real-time data processing
• Utilize Terraform and Databricks Asset Bundles for infrastructure management
• Leverage Cloud Platforms and CI/CD for deployment
• Provide technical leadership and mentor junior team members
Requirements
• 8+ years of experience in data engineering
• Strong expertise in Databricks, Delta Lake, and Unity Catalog
• Proficiency in PySpark and SQL
• Experience with Medallion Architecture
• Knowledge of Structured Streaming and Delta Live Tables
• Familiarity with Terraform and Databricks Asset Bundles
• Experience with Cloud Platforms and CI/CD
• Technical leadership and team mentoring skills