Data Engineering Lead
Indhu MRemoteLEADPosted 41m ago
Azure DatabricksPythonPySparkSQLAzure Data FactoryADLSApache SparkData Pipeline DevelopmentBatch Data ProcessingReal-time Data ProcessingADXAzure SQL DBAzure FunctionsAWSGlueKinesisEMRRedshiftLambda
About the Role
We are looking for an experienced Data Engineering Lead to design and develop scalable data engineering solutions using Azure Databricks, Python, PySpark, and SQL.
Key Responsibilities
• Design and develop end-to-end data pipelines
• Work on data engineering and cloud modernization projects
• Build and optimize scalable data solutions
• Provide technical guidance and support to the team
• Work on Azure and AWS cloud environments
Requirements
• 8–12 years of experience
• Strong experience in Azure Databricks, Python, PySpark, and SQL
• Hands-on experience with Azure Data Factory (ADF) and ADLS
• Good knowledge of Apache Spark and data pipeline development
• Experience in batch and real-time data processing
• Knowledge of Azure cloud services such as ADX, Azure SQL DB, and Azure Functions
• Experience in data architecture, performance tuning, and data optimization
• Good to have AWS experience (Glue, Kinesis, EMR, Redshift, Lambda)