Jobs → Data & Analytics → Lead Data Engineer

About the Role We are seeking a Lead Data Engineer with extensive experience in Databricks to join our team. This role involves designing, building, and optimizing scalable ETL/ELT pipelines and implementing Lakehouse solutions. Key Responsibilities • Design, build and optimize scalable batch and streaming ETL/ELT pipelines on Databricks using PySpark, Spark SQL, Delta Lake, Delta Live Tables, and Lakeflow • Architect and implement Lakehouse solutions using the Medallion pattern with Unity Catalog for governance • Build and maintain enterprise data warehouse models on various SQL platforms • Migrate legacy ETL and on-prem warehouses to modern cloud lakehouse platforms • Enable BI and reporting by delivering semantic layers and curated datasets • Integrate data from various sources using ADF, AWS Glue, Fivetran, Kafka, or similar tools • Implement data quality, observability, and testing frameworks • Apply CI/CD and DataOps practices • Optimize cost and performance across clusters, SQL warehouses, storage, and queries • Partner with architects, analysts, data scientists, and business stakeholders Requirements • 8–12 years of data engineering experience delivering large-scale ETL and data warehouse solutions • Strong, hands-on Databricks expertise in PySpark, Spark SQL, Delta Lake, Unity Catalog, Workflows/Jobs, DLT/Lakeflow, Databricks SQL, and performance tuning • Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or equivalent practical experience