Databricks Data/ETL Developer
Shaik SamirRemoteMIDPosted 18m ago
DatabricksPySparkSQLETLdata engineeringDatabricks AutoloaderStructured StreamingDelta LakeBronze-Silver-Gold Medallion ArchitectureSpark performance optimizationAzure servicesADLSIAMData modelingrow-level securityGitHubCI/CDFiveTran
About the Role
We are looking for a Databricks Data/ETL Developer to join our team. This role focuses on developing and maintaining data pipelines using Databricks and related technologies.
Key Responsibilities
• Develop and maintain Databricks data pipelines
• Build and optimize Delta Lake tables
• Support ETL, CI/CD, and daily data refresh processes
• Perform data reconciliation and validation
• Troubleshoot Databricks-related queries
Requirements
• Strong Databricks, PySpark, and SQL experience
• ETL and data engineering pipeline development skills
• Experience with Databricks Autoloader and Structured Streaming
• Knowledge of Delta Lake and Bronze-Silver-Gold Medallion Architecture
• Spark performance optimization skills
• Azure services experience, including ADLS and IAM
• Data modeling and row-level security experience
• GitHub and CI/CD experience
• Medicaid Data Warehouse experience is mandatory