Loading...
LeadFull-timePythonJavaAzureCI/CDGit
Salary Range
$114,750 - $155,250 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a Principal Azure Data Engineer (Databricks) to join Capco. This role involves architecting and leading the delivery of enterprise-grade data solutions using the Databricks platform within the Azure ecosystem.
Key Responsibilities
- →Lead the end-to-end delivery of Databricks-based solutions across Azure environments
- →Architect secure and scalable pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog
- →Develop and enforce engineering best practices across the data lifecycle
- →Engage with clients to define solution strategies and data governance frameworks
- →Mentor engineering teams and contribute to internal capability development initiatives
Requirements
- ✓Proven experience with the Databricks platform, including Unity Catalog, DeltaLake, and orchestration
- ✓Expertise in Python, PySpark, and distributed data processing frameworks
- ✓Extensive background in CI/CD pipeline development with tools like Azure DevOps, Jenkins, GitHub Actions
- ✓In-depth understanding of data lakehouse principles, data modelling, and GDPR-compliant design
- ✓Experience building robust, production-grade data pipelines from ingestion through to serving
