Kafka Interview Questions

Analogy categoryAnalogies

How Kafka Interviews Are Usually Structured

Kafka interviews typically progress through three tiers: foundational concept checks (what is a partition, what is a consumer group, what does a replication factor of 3 mean), scenario-based questions that probe understanding of trade-offs (how would you handle a slow consumer, what happens during a broker failure), and open-ended system design questions (design an event-driven order processing system, design a change-data-capture pipeline). Strong candidates are distinguished not by memorizing configuration names but by being able to explain the reasoning behind a default — for example, why acks=all alone doesn't guarantee no data loss without min.insync.replicas also being set appropriately.

🏏

Cricket analogy: It's like a cricket selection trial that starts with basic fitness tests, moves to net practice against varied bowling, and finishes with a simulated match situation — testing fundamentals, adaptability, and game-reading ability, just as Kafka interviews escalate from definitions to design.

Common Fundamentals and Trade-off Questions

Expect to be asked to explain partitions and ordering (Kafka guarantees order only within a partition, not across an entire topic), the difference between at-most-once, at-least-once, and exactly-once delivery semantics, and how consumer group rebalancing works when a consumer joins or leaves. A very common follow-up is 'how would you achieve exactly-once end-to-end,' which good candidates answer by distinguishing between Kafka's internal exactly-once (transactions between Kafka topics via the Streams API or transactional producer) and end-to-end exactly-once involving external systems, which usually requires idempotent writes keyed by a unique message identifier rather than relying on Kafka alone.

🏏

Cricket analogy: It's like explaining that a scorecard only guarantees the correct sequence of deliveries within a single over, not across the whole innings without cross-referencing overs — Kafka similarly guarantees order only within a partition, not across a topic.

System Design and Advanced Questions

Advanced interviews often present an open-ended prompt like 'design a real-time fraud detection pipeline using Kafka' and evaluate how you reason about partition key selection for even load distribution, how you'd handle a hot partition caused by a skewed key, whether you'd use Kafka Streams or a separate consumer application for stateful aggregation, and how you'd size the cluster (partition count, replication factor, retention) against expected throughput and durability requirements. Interviewers also probe failure scenarios directly: what happens if the partition leader dies mid-write, how does the controller elect a new leader from the in-sync replica set, and what data loss risk exists if min.insync.replicas is set lower than the replication factor.

🏏

Cricket analogy: It's like a captain designing a bowling rotation to avoid overusing one bowler (avoiding a hot partition) while ensuring a backup bowler (in-sync replica) is always ready to step in if the lead bowler gets injured mid-over, mirroring leader failover.

java

// A commonly asked live-coding prompt: implement a custom partitioner
// that hashes on a sub-field to avoid hot partitions from a skewed key
public class TenantAwarePartitioner implements Partitioner {
    @Override
    public int partition(String topic, Object key, byte[] keyBytes,
                          Object value, byte[] valueBytes, Cluster cluster) {
        int numPartitions = cluster.partitionCountForTopic(topic);
        String compositeKey = key + "-" + System.nanoTime() % 16; // spread hot tenant
        return Math.abs(Utils.murmur2(compositeKey.getBytes())) % numPartitions;
    }
}

When answering system design prompts, always state your assumptions out loud first (expected throughput, message size, durability requirements, latency budget) before proposing partition counts or replication factors — interviewers weight this reasoning process as heavily as the final answer.

A frequent trap: candidates say 'set acks=all for no data loss' without mentioning min.insync.replicas. If min.insync.replicas=1 with replication.factor=3, acks=all only waits for one replica, so losing that single broker after a write can still lose data — interviewers specifically listen for this nuance.

Kafka interviews typically escalate from terminology to trade-off scenarios to open-ended system design.
Ordering is guaranteed only within a partition, never across an entire topic, unless a single-partition topic is used.
Be ready to distinguish at-most-once, at-least-once, and exactly-once semantics with concrete configuration examples.
Exactly-once end-to-end typically requires idempotent downstream writes, not just Kafka transactions alone.
System design answers should state throughput, latency, and durability assumptions before proposing partition/replication numbers.
Hot partitions from skewed keys are a common design pitfall; custom partitioners or key salting are standard mitigations.
acks=all without an appropriately set min.insync.replicas does not fully protect against data loss on broker failure.