Data Engineer
Utkarsh MehtaRemoteMIDPosted 4h ago
Azure DatabricksApache SparkPySparkPythonSQLETLDelta LakeCI/CDGitLabAzure DevOpsData pipeline monitoringData quality
About the Role
We are looking for a skilled Data Engineer with 3-6 years of experience to join our team. This role requires strong hands-on expertise in data engineering technologies and tools.
Key Responsibilities
• Develop and maintain data pipelines using Azure Databricks, Apache Spark, PySpark, and Python
• Design and implement ETL/ELT processes and optimize data models
• Utilize Delta Lake for data storage and performance optimization
• Implement CI/CD practices using GitLab and Azure DevOps
• Monitor data pipelines and ensure data quality
Requirements
• 3-6 years of experience in data engineering
• Strong expertise in Azure Databricks, Apache Spark, PySpark, and Python
• Proficiency in SQL and ETL/ELT pipeline development
• Experience with Delta Lake, data modeling, and performance optimization
• Knowledge of CI/CD, GitLab, and Azure DevOps
• Experience in QSR, Retail, or Customer Analytics is preferred