Databricks Data/ETL Developer
Shiva VattikotiUnited StatesMIDPosted 1h ago
DatabricksPySparkSQLETLdata engineeringDatabricks AutoloaderStructured StreamingDelta LakeBronze-Silver-Gold Medallion ArchitectureData modelingSpark performance optimizationAzure servicesADLSIAMGitHubCI/CDFiveTran
About the Role
We are looking for a Databricks Data/ETL Developer to join our team. This role focuses on designing, developing, and maintaining Databricks data pipelines in the healthcare and Medicaid domain.
Key Responsibilities
• Design, develop, and maintain Databricks data pipelines
• Support ETL and CI/CD pipelines for data lake tables
• Manage daily data refresh and reconciliation processes
• Validate data accuracy and implement row-level security
• Automate daily dataset updates
Requirements
• Medicaid Data Warehouse experience is mandatory
• NY State client experience preferred, OR experience working with any other State client
• Strong Healthcare Medicaid Eligibility domain experience
• Hands-on experience with Databricks, PySpark, and SQL
• ETL and data engineering pipeline development
• Databricks Autoloader or Structured Streaming
• Delta Lake and Bronze-Silver-Gold Medallion Architecture
• Data modeling and Spark performance optimization
• Azure services, including ADLS and IAM
• GitHub and CI/CD experience
• FiveTran experience preferred
• Relevant Databricks certifications are a plus