Venkateswara Gade

Kafka Administrator with 6 years of IT experience specializing in Apache Kafka cluster administration, platform upgrades, security, monitoring, troubleshooting, and performance tuning across large-scale production environments. Hands-on experience managing 10+ production Kafka clusters across multiple data centers, performing rolling upgrades from Kafka 0.8.2 through 3.5.x with zero downtime, configuring replication, partitions, consumer groups, retention policies, and implementing enterprise security using SASL/SCRAM, SSL/TLS, Kerberos, and ACLs. Experienced in administering the wider Kafka ecosystem including Kafka Connect, Schema Registry, MirrorMaker 2 for cross-datacenter replication, and disaster recovery planning.

Strong operational experience with Prometheus, Grafana, Kafka Exporter, JMX, and Confluent Control Center for cluster monitoring, alerting, and observability. Proven ability to automate Kafka provisioning and operations using Ansible, Terraform, Shell and Python scripting. Additional expertise in ActiveMQ, Apache NiFi, Apache Camel, and Spring Integration.

Experienced in ZooKeeper administration and ZooKeeper-to-KRaft migration initiatives.

Kafka Administrator / Platform Engineer - Entain India - Hyderabad, India

(2023-03)

Administer, secure, monitor, and optimize enterprise Apache Kafka platforms across multiple production data centers, supporting business-critical applications processing billions of events daily.

• Install, configure, upgrade, and administer Apache Kafka clusters (10+ clusters, 500+ topics) across Dev, UAT, and Production environments spanning multiple data centers.
• Manage topics, partitions, replication factors, ISR, retention policies, consumer groups, and handle partition rebalancing and broker scaling operations.
• Led phased Kafka platform upgrades from Kafka 0.8.2 → 1.1.x → 3.5.2 using rolling upgrade procedures with zero or minimal business downtime.
• Support ZooKeeper-to-KRaft migration including KRaft deployments, validation in lower environments, and production readiness planning.
• Implement and maintain cluster security using SASL/SCRAM-SHA-512, SSL/TLS encryption, mTLS, ACLs, and role-based access control across all environments.
• Manage certificate lifecycle including generation, rotation, renewal, and revocation across production clusters.
• Monitor cluster health using Prometheus, Grafana, Kafka Exporter, and JMX — with alerting configured for consumer lag, under-replicated partitions, broker failures, and disk pressure.
• Troubleshoot production issues including ISR shrink, consumer lag spikes, consumer rebalance storms, disk pressure, JVM/GC tuning, network bottlenecks, and SSL handshake failures.
• Administer the wider Kafka ecosystem: Kafka Connect connectors for source/sink integrations, Schema Registry for schema evolution and compatibility enforcement, and MirrorMaker 2 for cross-datacenter replication and disaster recovery.
• Perform capacity planning and benchmarking using kafka-producer-perf-test and kafka-consumer-perf-test to validate cluster readiness for high-volume production workloads.
• Automate provisioning and routine operations using Ansible playbooks, Terraform modules, and Shell/Python scripting — reducing cluster provisioning time from days to hours.
• Develop automation scripts for topic management, ACL provisioning, consumer group offset management, rolling restarts, and configuration updates — reducing manual operations by 70%.
• Administer and upgrade Apache ZooKeeper clusters (migration to ZooKeeper 3.5.9) as part of Kafka platform modernization.
• Define and maintain backup, disaster recovery strategies, failover procedures, and operational runbook documentation for Kafka platforms.
• Perform JVM tuning and troubleshooting for Kafka brokers, resolving GC-related issues impacting consumer offset commits and message processing latency.
• Deploy and manage Kafka UI with RBAC capabilities for operational visibility and self-service cluster inspection.
• Developed a Kafka Administration Portal (Spring Boot + MongoDB) enabling self-service topic management, ACL management, and approval-based workflows.
• Support messaging platforms beyond Kafka including Apache ActiveMQ, Apache NiFi, Apache Camel, and Spring Integration.
• Drive migration strategies from legacy messaging (ActiveMQ) to Apache Kafka for improved throughput, scalability, and event replay capabilities.
• Manage Apache NiFi data flow pipelines for event routing between on-premise systems and AWS MSK clusters.
• Participate in on-call rotation for Kafka platform support, handling P1/P2 production incidents with defined SLAs.
• Collaborate with application, data engineering, and infrastructure teams to troubleshoot end-to-end messaging issues and ensure reliable event delivery.
• Conduct root cause analysis for production incidents and implement permanent preventive solutions.
• Mentor team members on Kafka administration best practices, producer/consumer patterns, and operational procedures.

Systems Engineer - Tata Consultancy Services

(2020-11 - 2023-02)

Supported production Kafka and messaging environments, performing deployment, monitoring, and incident management for distributed applications.

• Administered Linux servers hosting Kafka and middleware components, performing health checks, log analysis, and performance monitoring.
• Developed Shell scripts for automated deployments, log rotation, and routine operational tasks.
• Built and maintained REST APIs using Spring Boot with integration to Kafka and ActiveMQ messaging middleware.
• Performed production support, troubleshooting, and Root Cause Analysis using Linux, application logs, and middleware debugging techniques.
• Participated in on-call rotations, SLA management, and stakeholder communication during critical incidents.

B.Tech - Electronics & Communication Engineering - KL University (2016 - 2020)