Senior Data Engineer
Tharun kumarUnited StatesSENIORPosted 1h ago
DatabricksSnowflakeApache SparkPySparkSQLHealthcare Clinical DataHL7FHIRICD-10CPTETL/ELT Pipeline DevelopmentDelta LakeLakehouse ArchitectureUnity CatalogDatabricks WorkflowsAzureAWSHIPAAPHI SecurityData Governance
About the Role
We are looking for an experienced Senior Data Engineer with expertise in healthcare clinical data and strong hands-on experience with Databricks, Snowflake, Spark/PySpark, and SQL.
Key Responsibilities
• Develop and maintain ETL/ELT pipelines using Databricks, Snowflake, Spark/PySpark, and SQL
• Build scalable Lakehouse/Medallion architecture (Bronze, Silver, Gold) using Delta Lake
• Ingest, transform, and standardize clinical data from EHR/EMR systems
• Work with healthcare standards including HL7, FHIR, CCD, ICD-10, CPT, LOINC, SNOMED, and RxNorm
• Prepare curated datasets for predictive analytics and machine learning
• Implement data quality checks, monitoring, and performance optimization
• Ensure HIPAA compliance, PHI security, and healthcare data governance
• Collaborate with client stakeholders, data architects, data science teams, and offshore engineers
Requirements
• 6+ years of Data Engineering experience
• Strong expertise in Databricks, PySpark, Spark, SQL, and Delta Lake
• Hands-on experience with Unity Catalog and Databricks Workflows
• Strong Snowflake experience in data loading, transformation, modeling, and performance tuning
• Good understanding of healthcare clinical data and EHR/EMR systems
• Knowledge of HL7, FHIR, ICD-10, CPT, LOINC, SNOMED, and RxNorm
• Experience with Azure or AWS cloud platforms
• Understanding of HIPAA, PHI security, and data governance
• Strong communication and stakeholder management skills