Kafka Engineer/Administrator

Project overview

Hi Associate,

Greetings for the day,

We have a position as a Kafka Engineer/Administrator role in St. Louis, MO (or) Greenwood Village, CO. Please find the job description below for this position.

Role: Kafka Engineer/Administrator

Responsibilities

Kafka Administration & Ongoing Support (On-Prem + Cloud)

*AWS MSK Modernization (Parallel Workstream) **

CI/CD & Infrastructure as Code

Observability & Monitoring

Producer / Consumer Coordination & Teamwork

• Collaborate with the platform team to administer and operate Kafka clusters across Cloudera, Confluent Platform, and AWS MSK.
• Provide day-to-day operational support: incident response, on-call participation, root-cause analysis, and continuous reliability improvements.
• Manage topics, partitions, replication, retention, quotas, ACLs, and consumer groups.
• Tune brokers, producers, and consumers for throughput, latency, and reliability.
• Handle patching, upgrades, capacity planning, and cluster health/performance troubleshooting.
• Configure and maintain security: TLS/SSL, SASL, mTLS, IAM (for MSK), RBAC/ACLs, and encryption at rest and in transit.
• Partner with the team on ongoing workload-transition efforts from on-prem (Cloudera/Confluent) to AWS MSK.
• Contribute to transition strategies (e.g., MirrorMaker 2 / Confluent Replicator / cluster linking) with minimal downtime and no data loss.
• Validate topic parity, offsets, throughput, and data integrity before and after transitions.
• Help define rollback plans and run dry-runs in lower environments.
• Build and maintain Kafka/MSK infrastructure using Terraform.
• Manage automated deployments and configuration through GitLab CI/CD pipelines.
• Implement GitOps practices for topic, ACL, and cluster configuration management.
• Participate in code reviews and enforce version control and environment promotion (dev -> test -> prod).
• Build and maintain monitoring, alerting, and dashboards in Datadog for Kafka/MSK (broker metrics, consumer lag, partition health, JVM, throughput).
• Work with the team to establish SLAs/SLOs and proactive alerting to reduce incidents and mean-time-to-resolution.
• Correlate Kafka metrics with application and infrastructure telemetry.
• Act as a technical liaison between the Kafka platform team and producer/consumer application teams.
• Advise application teams on client configuration, serialization/schema management, partitioning strategy, idempotence, and exactly-once/at-least-once semantics.
• Support onboarding of new producers/consumers and troubleshoot client-side issues (rebalancing, lag, poison messages).
• Coordinate schedules, testing, and change windows with impacted teams.
• Share knowledge, document runbooks, and contribute to a strong team culture of collaboration and continuous improvement.

Requirements

• 5+ years in data/platform/infrastructure engineering, with 4+ years operating Apache Kafka in production.
• Hands-on administration experience with Cloudera and Confluent Platform.
• Production experience with AWS MSK (provisioning, configuration, IAM auth, networking).
• Experience supporting and operating Kafka in production, including on-call and incident response.
• Strong Terraform and GitLab CI/CD experience for infrastructure automation.
• Hands-on Datadog experience for monitoring, dashboards, and alerting.
• Deep understanding of Kafka internals: partitions, replication, ISR, offsets, consumer groups, and delivery semantics.
• Solid AWS fundamentals: VPC, security groups, IAM, KMS, CloudWatch, Private Link.
• Strong Linux, networking, and scripting skills (Bash, Python).
• Excellent communication and collaboration skills, with a track record of working effectively in a team.

Nice to have

Thanks & Regards

RAJU REDDY |Sr. Team Manager

CIS Technology Inc

Plano, TX 75074

Direct:

Job Facts: Contract W2 | Contract Independent | 24 Months | Hybrid | Depends on Experience

Skills: Kafka Engineer, Observability, Monitoring

Reference: Dice Id: 91114999 | Position Id: 9128395

• Experience with MirrorMaker 2, Confluent Replicator, or cluster linking for replication and workload transition.
• Schema Registry, Kafka Connect, and ksqlDB experience.
• Kafka Streams or Flink familiarity.
• Kubernetes / EKS experience (e.g., Strimzi).
• Certifications: Confluent Certified Administrator, AWS Solutions Architect/DevOps.
• Experience in a regulated or large enterprise environment.

Project details

St. Louis, MO (or) Greenwood Village, CO

24 Months

They are currently working on migrating all Cloudera Kafka from on prem to AWS. They need this person to handle to workload

• Start: flexible
• Duration: 24 Months
• Remote: hybrid
• Location: St. Louis, MO, US, US