Jobs → Data & Analytics → Azure Data Engineer

About the Role We are seeking an experienced Azure Data Engineer to design, develop, and maintain scalable data engineering solutions within the Microsoft Azure cloud environment. Key Responsibilities • Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) • Develop and optimize data processing workflows using Azure Databricks, PySpark, and Python • Build and maintain ETL/ELT pipelines to ingest, transform, and load data from multiple sources • Work with Azure Data Lake Storage Gen2 to store and manage structured and unstructured data • Develop complex SQL queries and optimize data transformations for performance • Implement data quality checks, validation rules, and error-handling mechanisms • Integrate data from databases, APIs, files, and other enterprise systems • Monitor pipeline performance, troubleshoot failures, and resolve data processing issues • Collaborate with data architects, analysts, developers, and business stakeholders to understand data requirements • Implement security, access controls, and data governance best practices • Use Git and CI/CD pipelines to manage code, version control, and deployments • Optimize data processing performance, scalability, and cloud resource utilization Requirements • Strong hands-on experience with Azure Data Factory (ADF) • Strong experience with Azure Databricks and Apache Spark • Proficiency in Python, PySpark, and SQL • Experience developing ETL/ELT pipelines and data integration workflows • Knowledge of Azure Data Lake Storage Gen2 • Experience working with relational databases and data warehousing concepts • Understanding of data modeling, data transformation, and data quality practices • Experience with Git, version control, and CI/CD processes • Strong analytical, problem-solving, and communication skills