Databricks Engineer
Vishal SharmaUnited StatesSENIORPosted 1h ago
DatabricksApache SparkPySparkPythonSQLAWSS3GlueAthenaLambdaStep FunctionsEventBridgeETLELTDimensional Data ModelingGitHub ActionsCI/CDTerraformCloudFormationDelta LakeApache IcebergAirflow
About the Role
We are looking for an experienced Databricks Engineer to build and optimize scalable data pipelines, migrate workloads from AWS EMR to Databricks, and support enterprise data analytics.
Key Responsibilities
• Build and optimize scalable data pipelines
• Migrate workloads from AWS EMR to Databricks
• Support enterprise data analytics
Requirements
• Experience with Databricks, Apache Spark & PySpark
• Proficiency in Python & Advanced SQL
• Knowledge of AWS: S3, Glue, Athena, Lambda, Step Functions & EventBridge
• Experience with ETL/ELT Pipelines & Dimensional Data Modeling
• Familiarity with GitHub Actions, CI/CD, Terraform/CloudFormation
• Understanding of Delta Lake, Apache Iceberg, Airflow
• Financial Services / Asset Management experience