Technical Lead – Data Engineering
Sampath MadanuUnited StatesSENIORPosted 10d ago
DatabricksAWSPySparkSpark SQLPythonSQLDelta LakedbtAirflowTerraformGitHub ActionsUnity CatalogAI-assisted development tools
About the Role
We are seeking a Technical Lead for Data Engineering with extensive experience in Databricks and AWS. This role involves designing scalable Lakehouse architectures and leading technical implementations.
Key Responsibilities
• Design and implement scalable Lakehouse architectures and enterprise-grade data platforms
• Develop complex batch, streaming, and event-driven data pipelines
• Implement automated data quality checks, end-to-end testing, and CI/CD integration
• Lead architecture reviews, code reviews, and technical governance
• Collaborate with Product Owners, Supply Chain teams, and AI/ML stakeholders
Requirements
• 15+ years of experience in data engineering
• Strong expertise in Databricks, AWS, PySpark, Spark SQL, Python, SQL, and Delta Lake
• Experience with dbt, Airflow, Terraform, GitHub Actions, or equivalent tools
• Knowledge of AI-assisted development tools such as GitHub Copilot, Amazon Q, Cursor, or Databricks Assistant
• Familiarity with Databricks Lakeflow, Mosaic AI, MLflow, AI Search, and AI/ML-ready data foundations
• Experience in Supply Chain, MedTech, SAP, ERP, MES, or PLM environments is a plus