Databricks Engineer
Rachel .KUnited StatesSENIORPosted 2h ago
DatabricksPythonPySparkSpark SQLAzureADLS Gen2Azure Data FactoryKey VaultMicrosoft Entra IDETLELTDelta LakeUnity CatalogLakeflow JobsLakeflow Spark Declarative PipelinesMedallion ArchitectureDimensional modelingCI/CDAzure DevOpsGitHub Actions
About the Role
We are looking for an experienced Databricks Engineer to join our team and build scalable, secure, and high-performance data engineering solutions using the Databricks Data Intelligence Platform.
Key Responsibilities
• Build scalable, secure, and high-performance data engineering solutions using the Databricks Data Intelligence Platform
• Design and implement batch and streaming ETL/ELT pipelines
• Work with Delta Lake, Unity Catalog, Lakeflow Jobs, and Lakeflow Spark Declarative Pipelines
• Implement Medallion Architecture (Bronze, Silver, Gold), dimensional modeling, and SCD Type 1/Type 2
• Lead technical design discussions, conduct code reviews, and mentor engineers
Requirements
• 10+ years of experience in Data Engineering, Data Analytics, or Data Warehousing
• 5+ years of hands-on experience with Databricks in production environments
• 5+ years of experience with Python, PySpark, and Spark SQL
• 3+ years of experience with Microsoft Azure data services, including ADLS Gen2, Azure Data Factory, Key Vault, and Microsoft Entra ID
• Strong experience building batch and streaming ETL/ELT pipelines
• Expertise in Delta Lake, Unity Catalog, Lakeflow Jobs, and Lakeflow Spark Declarative Pipelines
• Experience with Medallion Architecture (Bronze, Silver, Gold), dimensional modeling, and SCD Type 1/Type 2
• Hands-on experience with CI/CD using Azure DevOps or GitHub Actions
• Strong skills in performance optimization, data governance, troubleshooting, and pipeline monitoring
Preferred Qualification
• Databricks Certified Data Engineer Professional