Data Engineer
Ashok R.United StatesSENIORPosted 4d ago
Azure Data EngineeringSQLPySparkAPI IntegrationData ValidationProduction SupportAzure Data LakeREST APIsCSVSQLPySparkSynapse NotebooksADF PipelinesAPI BatchingError HandlingLoggingMonitoringReconciliationHIPAASecurityPrivacyGitAzure DevOpsCI/CDMonitoring/LoggingOnshore-Offshore TeamsTechnical DocumentationWritten/Verbal Communication
About the Role
We are seeking an experienced Data Engineer to support the migration of Power BI reporting and customer-success workflows to ChurnZero. This role requires strong Azure data engineering, SQL, PySpark, API integration, data validation, and production-support skills.
Key Responsibilities
• Design and maintain integrations between Azure Data Lake and ChurnZero using REST APIs, CSV imports, or approved integration methods
• Analyze existing Power BI transformations and migrate applicable logic to SQL, PySpark, Synapse notebooks, or ADF pipelines
• Map source data to ChurnZero accounts, contacts, attributes, events, custom fields, and custom tables
• Build incremental, restartable, and duplicate-safe pipelines with API batching, retries, error handling, logging, monitoring, and reconciliation
• Perform source-to-target validation including row counts, duplicates, required fields, NULL handling, and date formatting
• Support ChurnZero dashboards, segments, ChurnScores, Plays, Journeys, and alerts; monitor and troubleshoot production pipelines across ADF, Synapse, Spark, ADLS, and ChurnZero
• Protect healthcare and customer data according to HIPAA, security, and privacy requirements; coordinate testing, deployments, production support, and maintain mappings, data-flow diagrams, runbooks, and support documentation
Requirements
• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or related field, or equivalent experience
• 8+ years in data engineering, ETL/ELT, integration, or production data operations
• Strong hands-on experience with Azure Data Factory, ADLS Gen2, Azure Synapse/Synapse Spark, Key Vault, SQL, Python, PySpark, Spark SQL, and Bash
• Experience processing Parquet and Delta Lake data and building incremental ETL/ELT pipelines
• Experience with REST APIs including authentication, JSON, HTTP, pagination, batching, retries, and error handling
• Experience analyzing Power BI data models, Power Query transformations, or DAX calculations; strong data-validation, reconciliation, troubleshooting, and production-support skills
• Experience with Git, Azure DevOps, CI/CD, monitoring/logging, Onshore-Offshore teams, clear technical documentation, and strong written/verbal communication