Jobs → Data & Analytics → Database Testing

About the Role We are looking for a seasoned Data Engineering leader to focus on advanced Databricks testing, data quality engineering, and AI automation in the healthcare payer sector. Key Responsibilities • Lead end-to-end data pipeline testing using Databricks, PySpark, Spark SQL and Delta Lake. • Design and implement Data Quality as Code frameworks built on Great Expectations, dbt, Soda, or custom PySpark libraries. • Integrate data quality checks into CI/CD pipelines, transaction flows, and ingestion layers. • Conduct performance tuning for large-scale data processing and maintain reusable DQ code libraries. • Collaborate with cross-functional teams in an Agile or SAFe environment. Requirements • 10+ years of technical experience in Data Engineering, Architecture, or Data Quality Engineering. • Minimum 5 years of Databricks testing experience. • Deep hands-on expertise in Databricks, PySpark, Spark SQL, Delta Lake, and performance tuning. • Strong knowledge of Snowflake, Snowpark, Dynamic Tables, and data sharing. • Advanced proficiency in Python, PySpark, and SQL. • Proven experience building programmatic data quality frameworks with Great Expectations, dbt, Soda, or custom solutions. • Ability to architect and implement DQ as Code and integrate it into ingestion, transformation and CI/CD pipelines. • Generative AI and advanced automation skills. • Healthcare Payer domain knowledge (Medicare, Medicaid, Claims, HL7/FHIR, PHI/PII compliance). • Preferred certifications: Databricks Certified Data Engineer Professional, SnowPro Advanced.