Lead Data Engineer
Syed IshaqUnited StatesLEADPosted 2h ago
Azure Data EngineeringAzure DatabricksDatabricks Delta Live TablesUnity CatalogStructured StreamingAzure Data FactoryADLSSynapse AnalyticsSQLPythonScalaSpark SQLETLCloud data lake architectureQlik ReplicateKafkaHadoopApache SparkPower BIQlik SenseAI/MLNLPGenAIAgileKanban
About the Role
We are looking for a Lead Data Engineer to join our team, focusing on Azure and Databricks technologies.
Key Responsibilities
• Design and implement scalable data solutions using Microsoft Azure Data Engineering
• Develop and optimize ETL pipelines
• Work with Azure Databricks, Delta Live Tables, and Unity Catalog
• Manage data governance and lineage
• Create structured streaming solutions
Requirements
• Strong expertise in Microsoft Azure Data Engineering
• Advanced experience with Azure Databricks
• Knowledge of Databricks Delta Live Tables (DLT)
• Experience with Unity Catalog, data governance, and lineage
• Proficiency in Structured Streaming and scalable data pipelines
• Familiarity with Azure Data Factory (ADF), ADLS, and Synapse Analytics
• Strong SQL, Python, Scala, and Spark SQL skills
• ETL pipeline development and optimization experience
• Cloud data lake architecture and data migration knowledge
Candidates must be comfortable working onsite in Houston, TX.