Senior Data Engineer
Rabina BishtUnited StatesSENIORPosted 43m ago
ScalaApache SparkAWS EMRPythonSQLETLGitCI/CDDatabricksDelta LakeAWS S3AWS GlueAWS RedshiftAWS AthenaApache AirflowAWS Step Functions
About the Role
We are looking for an experienced Senior Data Engineer to join our team. This role focuses on designing, developing, and optimizing large-scale ETL pipelines using Scala, Apache Spark, and AWS EMR.
Key Responsibilities
• Design, develop, and optimize scalable ETL pipelines using Scala and Apache Spark on AWS EMR
• Build and maintain Python scripts and utilities for data workflows and automation
• Develop complex SQL queries for data transformation, validation, and performance optimization
• Work with Databricks notebooks, jobs, and Delta Lake
• Troubleshoot production pipeline issues and perform root-cause analysis
• Drive automation, code quality, reusable solutions, and engineering best practices
• Independently manage complex technical challenges and contribute to design discussions
Requirements
• 8+ years of experience in Data Engineering
• Strong hands-on experience with Scala and Apache Spark
• Hands-on experience with AWS EMR
• Proficiency in Python and SQL
• Strong experience in ETL development and large-scale data pipelines
• Experience with Git, CI/CD, and production support
Good to Have
• Databricks, Delta Lake, and Databricks Jobs
• AWS S3, Glue, Redshift, and Athena
• Apache Airflow or AWS Step Functions
• Financial Services / Banking domain experience
• Technical mentoring and design leadership experience