Python Azure Data Engineer
Amit Kumar UpadhyayUnited StatesSENIORPosted 10d ago
PythonSQLETLELTAzure CloudDatabricksApache SparkPySparkAzure Machine LearningAzure DevOpsGitHub ActionsJenkinsData governanceData securityData lineageUnity CatalogJavaSpringAI-powered development tools
About the Role
We are looking for an experienced Python Azure Data Engineer to build scalable, secure, and production-ready data solutions using Python, Azure, Databricks, and Apache Spark.
Key Responsibilities
• Design and optimize scalable data pipelines
• Develop high-performance ETL/ELT solutions
• Optimize Spark jobs and Databricks workloads
• Implement CI/CD automation and production deployments
• Ensure data security, governance, and compliance
• Collaborate with cross-functional teams in an Agile environment
Requirements
• 7+ years of hands-on Python, SQL, and ETL/ELT experience
• Strong experience with Azure Cloud, Databricks, and Apache Spark
• Expertise in PySpark, data pipelines, and large-scale data processing
• Experience with Azure Machine Learning (Azure ML)
• CI/CD pipelines using Azure DevOps, GitHub Actions, Jenkins, or similar tools
• Production monitoring, logging, alerting, and troubleshooting
• Data governance, security, access control, and data lineage
• Experience deploying solutions across Dev, QA, and Production environments