Databricks Engineer
Akola AravindUnited StatesSENIORPosted 2h ago
DatabricksPythonPySparkSQLAzureAWSGCPTerraformDelta SharingLakehouse Federation
About the Role
We are looking for a Databricks Engineer to join our team. This role focuses on the end-to-end data lifecycle, including analysis, design, and implementation of data solutions on the Databricks Data Intelligence Platform.
Key Responsibilities
• Design and implement data solutions on the Databricks Data Intelligence Platform
• Leverage Databricks, Python, PySpark, and SQL for data ingestion, transformation, modeling, and pipeline orchestration
• Lead solution design and set engineering standards for pipeline building, governance, and deployment
• Guide other data engineers in their work
Requirements
• 10+ years of professional experience in data engineering, data & analytics, or data warehousing
• 5+ years of hands-on experience working with Databricks in a data engineering capacity
• 5+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark
• 3+ years of data engineering experience on Microsoft Azure (e.g., ADLS Gen2, Azure Data Factory, Key Vault, Microsoft Entra ID)
Preferred Qualifications
• Active Databricks Certified Data Engineer Professional certification
• Active Databricks Certified Data Engineer Associate certification
• Experience with healthcare data (e.g., claims, EHR/clinical data, HL7/FHIR) and HIPAA/PHI handling
• Infrastructure as code experience with Terraform (Databricks and Azure providers)
• Experience preparing data for machine learning and generative AI use cases
• Familiarity with Delta Sharing and Lakehouse Federation