AWS & ETL Cloud Engineer
PAVAN BUnited StatesSENIORPosted 4d ago
AWSETLPythonPySparkSparkQlikIICSIDMCSQLRelational databasesNoSQL databasesLinuxShell scriptingControl-MAuto-SysCloudWatchELK StackElasticsearchLogstashKibanaBeatsCI/CDGitAgile
About the Role
We are seeking an experienced AWS & ETL Cloud Engineer Professional to join our team. This role involves developing and supporting cloud-based data pipelines and applications on AWS.
Key Responsibilities
• Develop and support cloud-based data pipelines and applications on AWS
• Utilize AWS services including S3, Glue, EMR, Lambda, Step Functions, CloudWatch, CloudTrail, SNS, SQS, EC2, VPC, RDS, and IAM
• Work with ETL and Data Integration tools such as Qlik, IICS, and IDMC
• Implement solutions using Python and/or PySpark/Spark
• Manage SQL, relational databases, and NoSQL databases
• Use Linux and shell scripting for various tasks
• Employ scheduling/orchestration tools like Control-M or Auto-Sys
• Monitor and troubleshoot using CloudWatch and/or the ELK Stack
• Apply CI/CD practices, Git/version control, and automated deployment processes
• Follow Agile software development methodologies
Requirements
• 12-15 years of experience in cloud engineering
• Hands-on experience with AWS services
• Proficiency in Python and/or PySpark/Spark
• Experience with ETL and Data Integration tools
• Strong SQL and database skills
• Familiarity with Linux and shell scripting
• Experience with scheduling/orchestration tools
• Monitoring and troubleshooting skills
• Knowledge of CI/CD practices and Agile methodologies