blog
Field notes from the Kafka on-call rotation.
Long-form posts on the patterns that cost teams their SLA — written by the engineers who built the playbooks that close those incidents before the pager fires.
- kafka on-call·
Why consumer lag spikes at 3am
Nightly rebalances, log retention cleanup, batched producer retries, and under-provisioned consumers — the four-forces pattern that lights up the pager every night, and the three-step checklist that closes it.
Read post →
- kafka on-call·
Why consumer rebalances stall your pipeline
Partition reassignments, broker bounces, group-size churn, and max.poll.interval.ms violations — the rebalance storm that stalls an otherwise healthy pipeline, and the three-step checklist that keeps the pipeline flowing.
Read post →
- kafka on-call·
Kafka broker memory pressure: the warning signs before an outage
Memory pressure rarely begins with an OOM. It starts with page-cache churn, an old gen that never comes back down, fetch waits tuned for a different workload, and one broker whose metrics drift away from its peers.
Read post →