Loading...
Mid LevelFull-timePythonAWSGCPAzureMachine LearningData ScienceCI/CD
Salary Range
$20,400 - $27,600 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a highly skilled Data Engineering with deep expertise in Databricks, ETL/ELT pipeline development. This role focuses on building scalable data solutions using Medallion architecture and ensuring data quality and compliance.
Key Responsibilities
- →Lead the design, development, and optimization of ETL/ELT pipelines using Databricks, Python, Spark, and Delta Lake.
- →Architect scalable data solutions using Medallion architecture (Bronze, Silver, Gold layers).
- →Design and implement data models and transformations using SQL and Python.
- →Build and maintain audit frameworks to ensure traceability, compliance, and data lineage.
- →Develop data quality monitoring and automated testing frameworks for pipeline reliability.
- →Perform data analysis to support operational data requests and user queries.
- →Collaborate with clinical data teams to analyze IRT/RTSM datasets.
- →Create and maintain dashboards and reports using BI tools.
- →Help manage CICD and automated code branching/deployment.
- →Ensure compliance with GxP, CDISC, and other regulatory standards.
- →Mentor junior engineers and promote engineering best practices.
Requirements
- ✓8–10 years of experience in data engineering, with leadership or team lead responsibilities.
- ✓Strong hands-on experience with Databricks, Apache Spark, and Delta Lake.
- ✓Advanced proficiency in SQL and Python for data transformation and automation.
- ✓Experience with ETL/ELT orchestration, job optimization, and performance tuning.
- ✓Proven experience designing and implementing audit, data quality, and testing frameworks.
- ✓Hands-on experience with IRT/RTSM clinical trial data systems.
- ✓Strong data analysis skills and ability to interpret complex datasets.
- ✓Experience with BI/reporting tools such as Power BI, Tableau, or Qlik.
- ✓Knowledge of clinical data standards (e.g., CDISC, SDTM, ADaM).
- ✓Experience with cloud platforms (Azure, AWS, or GCP) and CI/CD pipelines.
- ✓Bachelor’s degree in computer science, Engineering, or equivalent field or appropriate experience.
