Kafka

该职位来源于猎聘 The team member will be part of a support team responsible for maintaining Kafka clusters across all non-production and production environments. The role involves ensuring system reliability, onboarding consumers/producers, optimizing performance, automating regular tasks, and troubleshooting issues to maintain seamless Kafka operations.

Key Responsibilities

• Manage and maintain Kafka clusters in non-prod and prod environments
• Set up and configure Kafka clusters with ZooKeeper/KRaft
• Implement and manage various Kafka connectors
• Develop and maintain ksqlDB streams and tables
• Handle Kafka operations on Kubernetes
• Create and manage Docker images when not available in public registries
• Write and manage YAML files for configuration
• Implement CI/CD pipelines using GitLab
• Monitor Kafka environments using tools like Datadog
• Automate infrastructure management using Terraform
• Manage AWS services including security groups, load balancers, Route 53, networking, S3, and ECR
• Troubleshoot and resolve Kafka and infrastructure-related issues
• Adhere to incident, change, and problem management processes Qualifications
• Strong knowledge of Kafka architecture
• Hands-on experience with Kafka cluster setup using ZooKeeper/KRaft
• Proficiency in working with Kafka connectors and ksqlDB
• Experience with Kubernetes for Kafka deployment and management
• Ability to create Docker images and work with containerized environments
• Proficiency in writing YAML configurations
• Strong troubleshooting and debugging skills
• Experience with monitoring tools like Datadog
• Familiarity with GitLab CI/CD pipelines
• Proficiency in Terraform for infrastructure automation
• Strong understanding of AWS cloud services
• Experience with IT service management processes (incident, change, and problem management) Experiences
• 3+ years of experience managing Kafka clusters in production environments
• Experience setting up and maintaining Kafka on Kubernetes
• Hands-on experience with AWS cloud infrastructure
• Experience with CI/CD pipelines, infrastructure automation, and monitoring tools
• Certifications (Confluent, AWS) Language Proficiency in Mandarin Proficiency in English