AWS Data Engineer
Navya GrandheAustraliaSENIORPosted 4d ago
AWSData EngineeringAmazon SageMakerMLOpsPythonSQLPySparkAWS GlueEMRLambdaKinesisMSKAirflowS3Lake FormationAthenaRedshiftTerraformCloudFormationCDKCI/CDParquetIcebergDelta Lake
About the Role
We are looking for an experienced AWS Data Engineer with strong hands-on Amazon SageMaker to join our team.
Key Responsibilities
• Design and maintain scalable batch and streaming data pipelines using AWS Glue, EMR, Lambda, Kinesis/MSK, and Airflow
• Develop data lake and warehouse solutions using Amazon S3, Lake Formation, Athena, and Redshift
• Build ML-ready datasets and manage features using SageMaker Processing, Data Wrangler, and Feature Store
• Develop end-to-end ML workflows using SageMaker Pipelines, training, tuning, evaluation, and Model Registry
• Deploy and monitor ML models using SageMaker endpoints and CloudWatch
• Implement data governance, security, data quality, and lineage controls
• Automate infrastructure and deployments using Terraform, CloudFormation/CDK, and CI/CD pipelines
• Optimise data and ML workloads for performance, scalability, reliability, and cost
Requirements
• 7+ years of experience in Data Engineering, with strong hands-on AWS expertise
• Proven experience with Amazon SageMaker, including Pipelines, Processing, Feature Store, training jobs, and endpoints
• Strong proficiency in Python, SQL, and PySpark
• Extensive experience with AWS services such as S3, Glue, Athena, Redshift, Lambda, IAM, and KMS
• Experience with workflow orchestration using Airflow/MWAA or Step Functions
• Hands-on experience with Infrastructure as Code and CI/CD using Terraform, CloudFormation, or CDK
• Strong understanding of data modelling, partitioning, and columnar formats such as Parquet, Iceberg, or Delta Lake
• Excellent communication and collaboration skills