DQA Engineering Lead
Nithish JayaprakashRemoteLEADPosted 3h ago
DatabricksPySparkSpark SQLDelta LakeSnowflakeSnowparkPythonSQLDQxCShift-Left AutomationGreat Expectationsdbt TestsSodaCI/CDschema validationanomaly detectionGenerative AI
About the Role
We are seeking a DQA Engineering Lead with expertise in Databricks and Snowflake to lead data quality initiatives.
Key Responsibilities
• Implement and maintain data quality frameworks using tools like Great Expectations, dbt Tests, and Soda
• Develop and enforce Data Quality as Code (DQxC) practices
• Design and implement Shift-Left Automation strategies
• Conduct code reviews and performance tuning for data quality solutions
• Integrate data quality checks into CI/CD pipelines
Requirements
• Strong hands-on expertise in Databricks, PySpark, Spark SQL, and Delta Lake
• Deep understanding of Snowflake architecture and Snowpark (Python)
• Advanced Python, PySpark, and SQL skills
• Experience with Data Quality as Code (DQxC) and Shift-Left Automation
• Familiarity with data quality frameworks: Great Expectations, dbt Tests, Soda, or custom frameworks
• CI/CD integration, schema validation, and anomaly detection experience
• Data quality architecture, performance tuning, and code reviews
Preferred Qualifications
• Healthcare Payer domain experience: Medicare, Medicaid, Claims, HL7/FHIR, PHI/PII
• Databricks Data Engineer Professional or SnowPro Advanced certification
• SAFe / Agile Pod-based delivery experience