Senior Data Engineer – Databricks
Himanshu SinghUnited StatesSENIORPosted 1h ago
Databricks ETLPySparkSQLLakeflowStreaming Data PipelinesTerraformAWSAzureGCP
About the Role
We are actively looking for an experienced Senior Data Engineer with strong hands-on expertise in Databricks ETL, PySpark, SQL, Lakeflow, and Streaming Data Pipelines, preferably with recent Banking or Financial Services domain experience.
Key Responsibilities
• Architect, design, and develop scalable Databricks ETL pipelines
• Implement Lakeflow Connect, Auto Loader, and Lakeflow Declarative Pipelines
• Build real-time data processing solutions using Spark Structured Streaming
• Design Delta Lake solutions following Medallion Architecture principles
• Optimize Databricks workloads for performance, scalability, and cost efficiency
• Implement Unity Catalog, data governance, automated testing, and CI/CD standards
• Lead technical discussions, mentor engineers, and own end-to-end delivery
• Collaborate with stakeholders to translate business requirements into technical solutions
Requirements
• 8+ years of Data Engineering experience
• 5+ years of hands-on Databricks production experience
• Strong expertise in PySpark, SQL, Delta Lake, and ETL architecture
• Practical experience with Lakeflow, Auto Loader, and Structured Streaming
• Experience designing scalable enterprise data platforms
• Banking / Financial Services domain experience preferred
• Strong technical leadership, ownership, and communication skills
• Databricks Professional Certification is a plus