Jobs → Engineering → AWS S3 Python Migration Platform Engineer

About the Role We are seeking an AWS S3 Python Migration Platform Engineer to lead the design, build, and operation of a secure, scalable migration platform for moving object data from OBSv2 to AWS S3. This role focuses on reliable execution, automation, governance, monitoring, and production support for large-scale enterprise data migration. Key Responsibilities • Own the end-to-end migration workflow from OBSv2 source buckets to AWS S3 destination buckets. • Build and operate AWS-based orchestration using Step Functions, Lambda, Fargate, AWS Batch, DynamoDB, SQS, and rclone. • Support both manifest-based and dynamic migration approaches, including batching, sharding, task sizing, and retry handling. • Implement governance for bandwidth allocation, queue management, prioritisation, throttling, and job tracking. • Build observability through CloudWatch dashboards, logs, metrics, alerts, health checks, and progress reporting. Requirements • Experience with Dell ECS, OBSv2, or other S3-compatible enterprise object storage platforms. • AWS Serverless: Hands-on experience with AWS Lambda, Step Functions, SQS, SNS, EventBridge, API Gateway, SSM Parameter Store, and Secrets Manager. • AWS Compute and Containers: Strong working knowledge of ECS Fargate, AWS Batch, container task definitions, sidecar patterns, resource sizing, and runtime environment configuration. • AWS Storage: Deep understanding of Amazon S3, S3 Intelligent-Tiering, S3-compatible APIs, object metadata, prefix structures, large object transfers, and lifecycle considerations. • Object Storage Migration: Experience migrating large object datasets from on-premises or S3-compatible platforms such as Dell ECS/OBSv2 to AWS S3. • Rclone: Hands-on experience configuring and operating rclone for high-volume transfers, including transfers, checkers, retries, checksum validation, no-traverse behaviour, stats, logs, and bandwidth limiting. • Python Development: Strong Python skills for Lambda and Fargate services, including boto3, JSON processing, exception handling, logging, dataclasses, streaming files, and API integration. • DynamoDB: Ability to design and operate DynamoDB tables, keys, indexes, scans, updates, batch writes, Decimal handling, and state-management patterns. • Networking: Understanding of AWS Direct Connect, Transit Gateway, VPCs, private subnets, routing, S3 Gateway Endpoints, NAT Gateway scoping, and high-throughput network design.