Jobs → Operations → Kafka Administrator

About the Role We are migrating a portfolio of Apache Kafka clusters from on-premises platforms to AWS MSK. We are looking for a senior, hands-on Kafka engineer to own day-to-day Kafka administration across both environments, drive the migration, and coordinate with producer and consumer application teams. Key Responsibilities • Administer and operate Kafka clusters across Cloudera, Confluent Platform, and AWS MSK • Manage topics, partitions, replication, retention, quotas, ACLs, and consumer groups • Tune brokers, producers, and consumers for throughput, latency, and reliability • Handle patching, upgrades, capacity planning, and cluster health/performance troubleshooting • Configure and maintain security: TLS/SSL, SASL, mTLS, IAM (for MSK), RBAC/ACLs, and encryption at rest and in transit • Plan and execute migration of clusters and workloads from on-prem to AWS MSK • Design cutover strategies with minimal downtime and no data loss • Validate topic parity, offsets, throughput, and data integrity pre- and post-migration • Define rollback plans and run migration dry-runs in lower environments Requirements • 5+ years in data/platform/infrastructure engineering, with 4+ years operating Apache Kafka in production • Hands-on administration experience with Cloudera and Confluent Platform • Production experience with AWS MSK (provisioning, configuration, IAM auth, networking) • Proven experience migrating Kafka workloads to the cloud with minimal disruption • Strong Terraform and GitLab CI/CD experience for infrastructure automation • Hands-on Datadog experience for monitoring, dashboards, and alerting • Deep understanding of Kafka internals: partitions, replication, ISR, offsets, consumer groups, and delivery semantics • Solid AWS fundamentals: VPC, security groups, IAM, KMS, CloudWatch, PrivateLink • Strong Linux, networking, and scripting skills (Bash, Python) • Excellent communication and coordination skills to work across producer/consumer teams