Kafka Engineer
Adarsh T.United StatesSENIORPosted 1h ago
KafkaAWSMSKTerraformGitLabDatadogLinuxBashPython
About the Role
We are migrating a portfolio of Apache Kafka clusters from on-premises platforms to AWS MSK. We are looking for a senior, hands-on Kafka engineer to own day-to-day Kafka administration across both environments, drive the migration, and coordinate with producer and consumer application teams.
Key Responsibilities
• Own day-to-day Kafka administration across on-premises and cloud environments
• Drive the migration of Kafka clusters from on-premises to AWS MSK
• Coordinate with producer and consumer application teams for a smooth cutover
• Act as the technical point of contact between platform and application teams
Requirements
• 10+ years in data/platform/infrastructure engineering, with 4+ years operating Apache Kafka in production
• Hands-on administration experience with Cloudera and Confluent Platform
• Production experience with AWS MSK (provisioning, configuration, IAM auth, networking)
• Proven experience migrating Kafka workloads to the cloud with minimal disruption
• Strong Terraform and GitLab CI/CD experience for infrastructure automation
• Hands-on Datadog experience for monitoring, dashboards, and alerting
• Deep understanding of Kafka internals: partitions, replication, ISR, offsets, consumer groups, and delivery semantics
• Solid AWS fundamentals: VPC, security groups, IAM, KMS, CloudWatch, PrivateLink
• Strong Linux, networking, and scripting skills (Bash, Python)
• Excellent communication and coordination skills to work across producer/consumer teams