Kafka Engineer
Vivek UpadhyayUnited StatesSENIORPosted 7d ago
Apache KafkaAWS MSKTerraformGitLab CI/CDDatadogLinuxBashPythonAWSVPCIAMKMSCloudWatchSecurity GroupsPrivateLink
About the Role
We are looking for a Senior Kafka Engineer / Administrator with strong hands-on experience in Apache Kafka administration, cloud migration, and infrastructure automation. The ideal candidate will lead the migration of Kafka clusters from Cloudera and Confluent Platform to AWS MSK, leveraging Terraform and GitLab CI/CD while ensuring minimal disruption to production workloads.
Key Responsibilities
• 4+ years of hands-on Apache Kafka production experience
• Strong administration experience with Cloudera and Confluent Platform
• Hands-on AWS MSK provisioning, configuration, IAM authentication, and networking
• Experience migrating Kafka workloads from on-premises to AWS
• Strong Terraform and GitLab CI/CD skills
• Hands-on Datadog monitoring, dashboards, and alerting
• Deep understanding of Kafka partitions, replication, ISR, offsets, and consumer groups
• Strong AWS knowledge: VPC, IAM, KMS, CloudWatch, Security Groups, and PrivateLink
• Proficiency in Linux, Bash, Python, and networking
• Excellent communication and coordination with producer and consumer application teams
Preferred Skills
• MirrorMaker 2, Confluent Replicator, or Cluster Linking
• Schema Registry, Kafka Connect, and ksqlDB
• Kafka Streams or Apache Flink
• Kubernetes / EKS / Strimzi
• Confluent or AWS certifications
• Experience in large enterprise or regulated environments