Data QA Engineer
Lalit KumarUnited StatesSENIORPosted 12d ago
Azure Databricksdata pipeline testingtest automationPythonPySparkSQLAzure Data FactoryETLTestingPower BITableau
About the Role
We are looking for an experienced Data QA Engineer with strong expertise in Azure Databricks, data pipeline testing, and test automation to join our team.
Key Responsibilities
• Perform end-to-end data validation, ETL testing, and source-to-target reconciliation
• Test Azure Databricks workflows, Delta Lake, Delta Live Tables, and Unity Catalog
• Validate Azure Data Factory pipelines, monitoring, and CI/CD deployments
• Develop automated data validation scripts using Python, PySpark, and SQL
• Validate Bronze, Silver, and Gold layers within the Lakehouse/Medallion architecture
• Ensure data accuracy across reporting platforms such as Power BI and Tableau
• Collaborate with cross-functional teams to investigate defects and maintain data quality
Requirements
• 8+ years in Data QA/Testing, including 3+ years in the Healthcare Payer domain