AWS Data Engineer
Manoj PUnited StatesSENIORPosted 44m ago
AWS Data EngineeringAmazon S3AWS GlueAmazon RedshiftAmazon AthenaAWS LambdaSQLPythonPySparkApache SparkETLELTApache AirflowData LakesLakehouse ArchitectureData WarehousingDimensional ModelingGitCI/CDApache KafkaAmazon KinesisGenerative AIAmazon BedrockLLMExcelCSVSchema evolution
About the Role
We are looking for a highly skilled AWS Data Engineer with strong expertise in data pipeline development, distributed processing, SQL optimization, and AWS performance tuning. This role requires working independently as a technical consultant while mentoring engineering teams.
Key Responsibilities
• Design and implement data pipelines on AWS
• Optimize SQL, Spark, and PySpark workloads
• Develop and optimize ETL/ELT processes
• Implement data quality monitoring and recovery mechanisms
• Mentor engineering teams on best practices
• Troubleshoot and analyze performance issues
Requirements
• 12+ years of experience in AWS Data Engineering
• Expertise in AWS services including Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena & AWS Lambda
• Advanced SQL query writing and performance tuning skills
• Strong programming skills in Python and PySpark
• Experience with Apache Spark and distributed data processing
• Experience with ETL/ELT pipeline development and optimization
• Strong understanding of Apache Airflow and workflow orchestration
• Experience with Data Lakes, Lakehouse Architecture, Data Warehousing & Dimensional Modeling
• Expertise in Spark execution, partitioning, caching, joins, shuffle operations & resource optimization
• Knowledge of data skew, memory management, file sizing, compression & query execution plans
• Experience with Git, CI/CD pipelines, automated testing & production support
• Strong troubleshooting, root cause analysis & performance benchmarking skills
• Experience with data quality, monitoring, exception handling & recovery mechanisms
• Excellent communication, technical leadership, mentoring & problem-solving skills