Kafka in Production

Cluster Sizing and Disk Math

How to calculate broker count, disk requirements, and replication overhead before you write a single line of producer code.

Partition Strategy

How to choose partition keys, calculate partition counts, and handle the painful reality of repartitioning a live topic.

Producer Tuning and Idempotency

How to configure acks, batching, compression, and idempotent producers to get both throughput and delivery guarantees without guessing at defaults.

Consumer Group Patterns

How rebalancing works, when it destroys throughput, and the offset commit strategies that determine whether you replay or lose messages.

Schema Evolution

How to evolve Kafka message schemas safely using compatibility modes, Schema Registry, and the discipline to never break a downstream consumer.

Exactly-Once, the Honest Version

What Kafka's exactly-once semantics actually guarantee, where they end, and why your application logic still needs to handle duplicates.

Multi-Region Replication

How MirrorMaker 2 and Cluster Linking work, where active-active falls apart, and the operational discipline required to run Kafka across regions.

Upgrades Without Downtime

How to roll Kafka broker upgrades safely, manage version skew between brokers and clients, and navigate the KRaft migration without a maintenance window.

Cost Optimization

How tiered storage, compression, retention tuning, and cross-AZ traffic patterns determine your Kafka bill—and how to cut it without sacrificing reliability.

Incident Library

Ten real Kafka production incidents—the signs you missed, why they happened, and the exact response that resolved each one.