AWS Data Pipeline Engineer
Saloni BhagatUnited StatesSENIORPosted 11m ago
AWS LambdaStep FunctionsS3GlueRedshiftAthenaPythonPySparkSQLETLELTData LakesData WarehousingBatch Data PipelinesReal-Time Data PipelinesData ModelingPerformance TuningTechnical LeadershipDockerKubernetesData Governance
About the Role
We are looking for an AWS Data Pipeline Engineer to join our team in the investment banking domain. This role involves designing and implementing data pipelines using AWS services and Python.
Key Responsibilities
• Design and implement data pipelines using AWS Lambda, Step Functions, S3, Glue, Redshift, and Athena
• Develop and maintain ETL/ELT processes for data lakes and data warehousing
• Create batch and real-time data pipelines
• Perform data modeling and performance tuning
• Provide technical leadership and mentorship
Requirements
• 13+ years of experience in data engineering
• Strong experience with AWS services including Lambda, Step Functions, S3, Glue, Redshift, and Athena
• Proficiency in Python, PySpark, and SQL
• Experience with ETL/ELT, data lakes, and data warehousing
• Knowledge of batch and real-time data pipelines
• Understanding of data modeling and performance tuning
• Experience with Docker, Kubernetes, and data governance is preferred