Senior Data Engineer
Sahil DhimanUnited StatesSENIORPosted 1h ago
Amazon RedshiftDatabricksPySparkSpark SQLPythonSQLDelta LakeApache SparkETLELTData modelingCI/CDInfrastructure as Code
About the Role
We are looking for an experienced Senior Data Engineer to join our team. This role focuses on migrating data pipelines and workloads from Amazon Redshift to Databricks Lakehouse Platform.
Key Responsibilities
• Migrate data pipelines, data models and reporting workloads from Redshift to Databricks
• Build scalable batch and streaming pipelines
• Optimize Spark jobs, Delta tables and SQL workloads
• Implement data quality and reconciliation frameworks
• Support migration cutover and documentation
Requirements
• Strong hands-on expertise in cloud data engineering and large-scale data warehouse migrations
• Experience with Amazon Redshift and Databricks
• Hands-on expertise in PySpark, Spark SQL and Python
• Strong SQL development and query optimization skills
• Experience with Databricks, Delta Lake and Apache Spark
• ETL/ELT pipeline migration and modernization experience
• Redshift-to-Databricks migration experience
• Data modeling – Star Schema, Snowflake Schema and Dimensional Modeling
• Data validation, reconciliation and performance tuning
• CI/CD and Infrastructure as Code experience