Loading...
Mid LevelFull-timePythonRustAzureAgileCI/CDGit
Salary Range
$102,000 - $138,000 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are seeking to hire a Data Engineer to join our team at CTEC, a leading technology firm providing services to the U.S. Federal Government. This role focuses on developing and maintaining scalable data pipelines and cloud-based data platforms.
Key Responsibilities
- →Design, develop, and maintain scalable ETL pipelines and data workflows
- →Build, optimize, and maintain data processing solutions using Azure Databricks
- →Support phased data migration from legacy databases to Azure Databricks environments
- →Implement layered lakehouse data architectures in Databricks
- →Develop data processing notebooks and distributed data transformations using Python and PySpark
- →Develop data validation and testing processes to ensure data accuracy
- →Integrate Databricks data platforms with analytics and reporting tools
- →Support data governance initiatives including metadata management and data security
- →Maintain source control and CI/CD pipelines for Databricks workflows
- →Provide ongoing support for Databricks workflows and troubleshoot data processing issues
- →Mentor junior data engineers and contribute to documentation
Requirements
- ✓At least seven (7–9+) years of experience in data engineering, ETL development, or large-scale data integration
- ✓Strong experience designing and developing ETL pipelines in Azure Databricks
- ✓Strong proficiency in SQL and Python, with hands-on experience using PySpark
- ✓Experience working with cloud-based data platforms and data lakes
- ✓Experience implementing layered lakehouse data architectures
- ✓Familiarity with Spark-based big data processing frameworks
- ✓Experience supporting data migration to Databricks-based platforms
- ✓Experience integrating Databricks platforms with business intelligence tools
- ✓Familiarity with data governance and data security best practices
- ✓Experience with source control and CI/CD pipelines
- ✓Working knowledge of SDLC and Agile delivery methodologies
- ✓Bachelor’s degree in Computer Science, Engineering, or a related technical discipline
- ✓Must be a U.S. citizen and able to obtain Public Trust clearance.
