Loading...
Senior Data Engineer - Lakehouse & Data Engineering Frameworks - REF5085J at DeutscheTelekomITSolutions
DeutscheTelekomITSolutionsBudapest, hu·4w ago
SeniorFull-timePythonRailsAzureCI/CD
Salary Range
$42,432 - $57,408 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a Senior Data Engineer to build and operate scalable data ingestion and change data capture (CDC) capabilities on our Azure-based Lakehouse platform. This role involves developing pipelines in Azure Data Factory and Databricks, while also applying professional software engineering practices to mature our engineering approach.
Key Responsibilities
- →Design and develop ingestion pipelines using Azure Data Factory and Databricks services
- →Implement and operate CDC patterns, including late arriving data and reprocessing strategies
- →Structure and maintain bronze and silver Delta Lake datasets with schema enforcement and performance tuning
- →Build transformation-ready datasets with stable schemas and contracts for analytics engineers
- →Ingest data in a batch-first approach and help evolve patterns toward streaming where needed
- →Develop and maintain Python-based ingestion/CDC components as production-grade software
- →Apply engineering best practices including code reviews, testing, and documentation
- →Establish and improve CI/CD pipelines for data engineering code
- →Drive reuse via shared libraries and reference implementations
- →Implement logging, metrics, and observability for data pipelines
- →Troubleshoot distributed processing and production issues
- →Mentor other engineers and promote consistent engineering practices
- →Contribute to architectural discussions and data governance standards
Requirements
- ✓Experience with Azure Data Factory and Databricks
- ✓Strong Python programming skills
- ✓Knowledge of data engineering principles and practices
- ✓Experience with CI/CD pipelines and software engineering best practices
- ✓Understanding of Delta Lake and data lakehouse architectures
- ✓Familiarity with observability tools and metrics
- ✓Experience with troubleshooting distributed systems
- ✓Ability to mentor other engineers and contribute to technical standards
- ✓Understanding of data governance and security requirements
Free to sign up. Takes 30 seconds.
