Backend Python Engineer
Sumit MadneUnited StatesSENIORPosted 1h ago
PythonDatabricksTerraformData MigrationPySparkApache SparkSQLETLAWSS3IAMEC2GlueRDS
About the Role
We are looking for an experienced Senior Backend Python Engineer with strong Databricks, Terraform, and Data Migration expertise to join our engineering team. This role focuses on managing Databricks environments, automating cloud infrastructure, and executing large-scale database migrations.
Key Responsibilities
• Design, configure, deploy, and administer Databricks environments, including clusters, workflows, Delta Lake, and Unity Catalog
• Build and automate infrastructure provisioning using Terraform and Infrastructure as Code (IaC)
• Lead end-to-end database and data platform migrations, including schema migration, ETL modernization, data validation, and performance tuning
• Develop and optimize scalable data pipelines using Python, PySpark, Apache Spark, and SQL
• Implement data ingestion, transformation, and migration workflows
• Work with AWS services such as S3, IAM, EC2, Glue, and RDS
• Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency
• Ensure data quality, security, and HIPAA compliance when handling healthcare datasets
• Troubleshoot infrastructure issues, pipeline failures, and data inconsistencies
• Independently own technical deliverables while collaborating with engineering teams
Requirements
• Hands-on production experience managing Databricks environments
• Experience automating cloud infrastructure using Terraform
• Experience executing large-scale database and data platform migrations
• Healthcare data experience with clinical, claims, patient, provider, or EHR/EMR datasets
• Understanding of HIPAA and PHI compliance
• Strong technical ownership, troubleshooting capabilities, and ability to independently drive initiatives
• Experience with Python, PySpark, Apache Spark, SQL, and ETL/ELT pipelines
• Working knowledge of AWS services (S3, IAM, EC2, Glue, RDS, cloud networking)