Why a Quick Reference Matters
Kafka's operational surface area is large enough that even experienced engineers keep a cheat sheet handy for topic management, consumer group inspection, and the handful of configuration properties that matter most day to day. This reference consolidates the CLI commands and settings you'll reach for repeatedly — creating and describing topics, checking consumer lag, and the core producer/consumer/broker properties — so you don't have to re-derive flag names under pressure during an incident.
Cricket analogy: It's like a fielding captain keeping a laminated card of field placements for different bowlers in their pocket — not because they don't know the game, but because recalling the exact setup instantly during a tense over saves precious seconds, just as a Kafka cheat sheet does during an incident.
Essential CLI Commands
The core operational commands revolve around kafka-topics.sh for topic management, kafka-console-producer.sh/kafka-console-consumer.sh for quick manual testing, and kafka-consumer-groups.sh for inspecting and resetting consumer lag — the single most useful command during an on-call incident, since it shows exactly how far behind each partition's consumer is. In newer Kafka versions running in KRaft mode (no ZooKeeper), the same tools work but you point --bootstrap-server at a broker rather than --zookeeper at a ZooKeeper ensemble, since ZooKeeper is no longer part of the architecture.
Cricket analogy: It's like a team analyst pulling up a player's real-time strike rate and run rate on a tablet mid-innings — kafka-consumer-groups.sh gives that same instant 'how far behind are we' snapshot for consumer lag during an incident.
# Topic management
kafka-topics.sh --bootstrap-server localhost:9092 --create --topic events --partitions 6 --replication-factor 3
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic events
kafka-topics.sh --bootstrap-server localhost:9092 --alter --topic events --partitions 12
kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic events
# Manual produce/consume for testing
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic events
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --from-beginning
# Consumer group inspection (the most-used incident command)
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group order-service
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group order-service \
--topic events --reset-offsets --to-earliest --executeKey Configuration Properties and Formulas
A handful of properties account for most of Kafka's day-to-day tuning surface: acks, enable.idempotence, and min.insync.replicas for producer durability; auto.offset.reset, enable.auto.commit, and max.poll.records for consumer behavior; and retention.ms, replication.factor, and min.insync.replicas at the topic level. Two formulas are worth memorizing: consumer lag is simply (latest offset in the partition) minus (last committed offset for that consumer group), and the maximum number of broker failures a topic can survive without becoming unavailable for writes is replication.factor minus min.insync.replicas.
Cricket analogy: It's like calculating a required run rate — runs needed divided by balls remaining — a simple formula every commentator recalls instantly; consumer lag is just as simple: latest offset minus committed offset.
min.insync.replicas is a topic-level (or broker default) setting, not a producer setting — it works together with the producer's acks=all to define durability. A common production baseline is replication.factor=3 with min.insync.replicas=2, tolerating one broker failure while still accepting writes.
kafka-consumer-groups.sh --reset-offsets is destructive and immediately changes what a live consumer group will read next; always run it with --dry-run first (the default without --execute) to preview the new offsets before committing to --execute.
- kafka-topics.sh handles topic creation, description, partition increases, and deletion.
- kafka-consumer-groups.sh --describe is the fastest way to check per-partition consumer lag during an incident.
- In KRaft mode, use --bootstrap-server instead of the legacy --zookeeper flag across all CLI tools.
- Consumer lag = latest partition offset minus the consumer group's last committed offset.
- A topic tolerates (replication.factor − min.insync.replicas) broker failures while still accepting writes.
- A common production baseline is replication.factor=3 with min.insync.replicas=2.
- Always dry-run kafka-consumer-groups.sh --reset-offsets before adding --execute, since it's a destructive operation.
Practice what you learned
1. Which command is most useful for diagnosing consumer lag during an incident?
2. How is consumer lag calculated?
3. What flag replaces the legacy --zookeeper flag across Kafka CLI tools in KRaft mode?
4. With replication.factor=3 and min.insync.replicas=2, how many broker failures can a topic tolerate while still accepting writes?
5. Why should you run kafka-consumer-groups.sh --reset-offsets without --execute first?
Was this page helpful?
You May Also Like
Kafka Performance Tuning
Practical tuning levers on the producer, consumer, and broker sides for maximizing Kafka throughput and minimizing latency.
Kafka Interview Questions
A structured walkthrough of common Kafka interview topics, from fundamentals to system-design-level questions, with the reasoning interviewers expect.
Kafka vs Other Message Queues
How Kafka's log-based, partitioned architecture compares to traditional brokers like RabbitMQ, cloud queues like AWS SQS, and newer systems like Apache Pulsar.
Related Reading
Related Study Notes in DevOps
Browse all study notesNginx Study Notes
DevOps · 30 topics
DevOpsAnsible Study Notes
DevOps · 30 topics
DevOpsAdvanced Kubernetes Study Notes
Kubernetes · 30 topics
DevOpsAdvanced Bash Scripting Study Notes
Bash · 30 topics
DevOpsDocker & Kubernetes Study Notes
YAML · 40 topics
DevOpsCI/CD Tools & Pipelines Study Notes
YAML · 37 topics