Senior Data Engineer
Veera Venkata Krishna PinniboinaUnited KingdomSENIORPosted 4h ago
PythonSQLDatabricksPySparkApache SparkDBTData ModellingDelta LakeAzure Cloud Data PlatformData quality frameworksData pipeline developmentData migration
About the Role
We are looking for an experienced Senior Data Engineer with strong Databricks expertise to drive the design, development, and migration of scalable data pipelines on a modern Azure Cloud Data Platform.
Key Responsibilities
• Design and build scalable, production-grade data pipelines.
• Develop ingestion, transformation, and curated data layers.
• Optimize large-scale data processing using PySpark and SQL.
• Implement data transformations and modelling using DBT.
• Design and maintain reliable Delta Lake architectures.
• Establish data quality standards and automated validation frameworks.
• Build secure, scalable, and cost-efficient solutions on Azure.
• Collaborate with cross-functional teams to deliver robust data solutions.
Requirements
• Strong experience in Python and SQL
• Hands-on expertise in Databricks, PySpark, and Apache Spark
• Experience with DBT and Data Modelling
• Strong understanding of Delta Lake architecture
• Azure Cloud Data Platform experience
• End-to-end data pipeline development and migration
• Data quality frameworks, validation, and monitoring