Lead Data Engineer
Aldrict LaminUnited StatesLEADPosted 1h ago
DatabricksPySparkDelta LakeSQLETLAWSUnix/LinuxInformatica
About the Role
We are seeking an experienced Lead Data Engineer with strong expertise in Databricks, PySpark, Delta Lake, and advanced SQL to design and deliver scalable data engineering solutions.
Key Responsibilities
• Design and develop scalable data pipelines using Databricks, PySpark, and Delta Lake
• Build, optimize, and maintain ETL/ELT workflows
• Lead ETL design, development, optimization, and migration initiatives
• Perform complex data transformations, data quality checks, and reconciliation
• Optimize SQL queries and pipeline performance
• Collaborate with business stakeholders, architects, and onsite/offshore teams
• Support production deployments, monitoring, and issue resolution
Requirements
• 10+ years of experience in Data Engineering and ETL
• 8+ years of experience coordinating with onsite/offshore teams and business stakeholders
• Strong hands-on experience with Databricks, PySpark, Notebooks, and Delta Tables
• Advanced SQL skills, including query optimization and analytics
• Strong understanding of ETL/ELT, data warehousing, data modeling, and pipeline architecture
• Exposure to AWS cloud services
• Unix/Linux shell scripting experience
• Experience in hybrid delivery environments
Good to Have
• Informatica IDMC and PowerCenter
• Experience migrating legacy ETL workflows from Informatica to Databricks