# 🚀 Throughput, latency and backpressure compared

## Learning objectives

- Explain how batching, compression and sequential disk writes give Kafka its throughput ceiling.
- Explain how prefetch and per-message acknowledgements shape RabbitMQ's latency floor and in-flight limit.
- Size prefetch and consumer concurrency with Little's Law instead of guessing.
- Predict which broker saturates first for a given volume and latency target, and say why.

## Quick review

Last lesson: ordering is a property of a container, not a broker. A Kafka partition orders; a RabbitMQ queue with one consumer orders. The key decides which container an event lands in.

Today we reuse that idea from the other side. **The number of containers is also your parallelism budget.** More partitions means more parallel batches. More queues means more parallel consumers. That single sentence explains most of the throughput difference you will measure.

## Theory

### Kafka: throughput comes from amortization

Kafka writes to an append-only log. Writes are sequential, and sequential I/O on a single spinning disk beats random I/O by orders of magnitude. Reads come from the page cache and go out through zero-copy `sendfile`. Neither path needs a per-message index lookup, which is why read cost stays flat as the log grows to terabytes.

The producer then amortizes everything else over a batch:

- `batch.size` is the byte ceiling per partition.
- `linger.ms` is the time ceiling. A record waits up to this long for company.
- Compression (lz4, zstd) is applied per batch, so bigger batches compress better.

The trade is explicit: `linger.ms` is latency you pay on every single record. On the consumer side, `fetch.min.bytes` and `fetch.max.wait.ms` do the same job in reverse.

### RabbitMQ: latency comes from push and small units of work

RabbitMQ pushes a message to a consumer, waits for an ack, and repeats. Every message is routed, delivered, tracked and acknowledged individually. That per-message bookkeeping is expensive at volume, but it is also why a single message can cross the broker in well under a millisecond when the queue is empty.

You cannot batch your way out of this. The lever is `basic.qos` prefetch: how many unacknowledged messages a consumer may hold. Prefetch 1 gives strict serialization and the smallest redelivery window. Prefetch 250 parks 250 messages in one consumer's memory and, if that consumer dies, redelivers all 250. Quorum queues add a Raft majority round trip to every publish and ack.

| Dimension | Kafka | RabbitMQ |
| --- | --- | --- |
| Unit of work | batch of records in a partition | one message per delivery and ack |
| Throughput lever | partitions, batch size, compression | queues, consumers, prefetch |
| Latency floor | set by `linger.ms` (single-digit ms) | sub-millisecond to a few ms when shallow |
| Slow consumer shows as | consumer lag | queue depth and unacked count |
| Buffer limit | retention time and size | memory and disk alarms |

### Backpressure: pull versus push

This is the part that surprises people in production.

RabbitMQ applies backpressure explicitly. A slow consumer grows the queue, queue depth eats memory, the memory high watermark (default 0.6 of RAM) or `disk_free_limit` trips, and the broker blocks publishers. The client sees `connection.blocked`. Your producers slow down, whether you planned for it or not. It is crude but honest: nothing is silently lost.

Kafka consumers pull. A slow consumer cannot push back on the broker, because the broker never pushes. The consumer simply accumulates lag, and the log absorbs the difference until `retention.ms` or `retention.bytes` deletes it. So ==Kafka's backpressure signal is data expiring before it is read==, which is far quieter than a blocked connection. Producer-side backpressure is separate: `buffer.memory` fills and sends block for up to `max.block.ms`.

## Worked examples

### Example 1: sizing prefetch with Little's Law

In-flight messages equal throughput multiplied by latency: $L = \lambda \times W$.

You need 6,000 msg/s across 24 consumers, so 250 msg/s per consumer. p99 processing time is 40 ms.

$$
L = 250 \times 0.040 = 10
$$

Prefetch must be at least 10, and you need roughly 10 concurrent workers per consumer to feed them. Compare the naive setup: prefetch 250 with one thread gives $1/0.040 = 25$ msg/s no matter what. You would be 10x short while holding 250 unacked messages as a redelivery bomb.

### Example 2: what `linger.ms` actually buys you

Producer volume is 15,000 messages/s at 1 KB, so 15 MB/s. With 6 partitions, each partition sees 2.5 MB/s. `batch.size` is 1 MB, `linger.ms` is 5.

Filling 1 MB at 2.5 MB/s takes 400 ms, so the size ceiling never fires. The actual batch is $2.5 \text{ MB/s} \times 0.005\text{ s} = 12.5$ KB, about 12 records.

You are paying per-batch overhead 200 times a second. Raise `linger.ms` to 20 if your latency budget allows: batches grow to 50 KB, overhead drops 4x, and lz4 starts finding real redundancy. `batch.size` is a ceiling, `linger.ms` is the dial at moderate volume.

### Example 3: which one saturates first

Workload: 40,000 events/s at 2 KB, so 80 MB/s, with a p99 end-to-end target of 200 ms.

Kafka: spread over 12 partitions, each writes 6.7 MB/s sequentially, comfortable for one node. Latency is `linger.ms` plus one `acks=all` round trip plus fetch, so tens of milliseconds. Headroom is real, and if you lose it you add partitions and brokers.

RabbitMQ: 40,000 msg/s with per-message acks on quorum queues needs many queues and connections, and every publish pays a majority round trip. Any consumer stall grows the queue, raises memory, and triggers flow control that blocks your publishers. p99 degrades long before the mean does.

Verdict: sustained 80 MB/s with a 200 ms budget belongs on Kafka. Flip the numbers, 200 events/s with a 5 ms budget, and RabbitMQ wins easily because nothing waits for a batch.

> [!NOTE]
> These figures are order-of-magnitude reasoning, not benchmarks. Measure your own message size, replication factor and hardware before committing.

## Discussion & reflection prompts

1. Your RabbitMQ publishers got `connection.blocked` during a consumer deploy. What did the broker protect you from, and what did it hide?
2. Your Kafka consumer group is 6 hours behind and retention is 24 hours. What alarm would you rather have than lag, and why is it hard to build?
3. Where in your system would a 20 ms `linger.ms` be invisible, and where would it break an SLA?

## Expert corner

- Benchmark with realistic depth. An empty RabbitMQ queue answers in microseconds. Once depth grows into the millions, a single queue's Erlang process becomes the bottleneck, while Kafka's read cost does not move.
- Report p99, never the mean. Batching and backpressure both live in the tail.
- Set `linger.ms` to roughly a tenth of your latency budget. Below that you are buying throughput you already have; above it you are burning SLA.
- Treat prefetch as a memory and risk budget, not a speed dial. Throughput comes from concurrency and consumer count.
- `acks=all` and quorum queues both add a replication round trip. Budget for it in the latency target rather than discovering it in production.
- Watch two different signals: consumer lag for Kafka, queue depth plus unacked count for RabbitMQ. They fail in opposite directions.

## Summary & next steps

Kafka buys throughput by amortizing cost over batches and sequential writes, and pays for it in a latency floor set by `linger.ms`. RabbitMQ buys low latency with per-message push and ack, and pays for it in per-message overhead and a single-queue ceiling. Backpressure differs just as sharply: RabbitMQ throttles the producer, Kafka lets the consumer fall behind into the retention window.

Next up: routing and topology. You will see how exchanges and bindings compare with topics and partitions, and how routing complexity changes the throughput numbers we just built.

```flashcard
[
  { "front": "What gives Kafka its high throughput?", "back": "Sequential log appends, plus batching and compression that amortize cost over a whole batch instead of one record." },
  { "front": "Which two producer settings control Kafka batching?", "back": "batch.size (byte ceiling per partition) and linger.ms (time ceiling)." },
  { "front": "What does linger.ms cost you?", "back": "Latency: every record waits up to that long for a batch to fill." },
  { "front": "What controls in-flight messages in RabbitMQ?", "back": "Prefetch (basic.qos): the maximum unacknowledged messages a consumer may hold." },
  { "front": "High prefetch with a single-threaded consumer: what happens?", "back": "Throughput stays at 1/processing time, but many messages sit unacked and all get redelivered if the consumer dies." },
  { "front": "What is Kafka's backpressure signal on the consumer side?", "back": "Consumer lag. The log absorbs it silently until retention expires and the data is deleted." },
  { "front": "How does RabbitMQ push back on producers?", "back": "Memory high watermark or disk_free_limit trips flow control and blocks publishers (connection.blocked)." },
  { "front": "Little's Law as used for prefetch sizing?", "back": "In-flight = throughput x latency (L = lambda x W)." },
  { "front": "Why does Kafka read cost stay flat as the log grows?", "back": "Consumers fetch sequentially by offset; there is no per-message index scan or deletion." },
  { "front": "Which settings add a replication round trip to publish latency?", "back": "acks=all with min.insync.replicas in Kafka, and publisher confirms on RabbitMQ quorum queues." },
  { "front": "When does RabbitMQ beat Kafka on latency?", "back": "Low to moderate volume with small messages, when per-message delay dominates and you cannot afford a batching wait." },
  { "front": "What does fetch.max.wait.ms do?", "back": "Caps how long a Kafka consumer waits to reach fetch.min.bytes before returning a smaller batch." }
]
```

Cloze check: Kafka amortizes cost per {{c1::batch}} rather than per message. RabbitMQ's prefetch caps {{c2::in-flight unacknowledged messages}} per consumer. Kafka consumers are {{c3::pull}}-based, so a slow consumer shows up as {{c4::consumer lag}} instead of throttling producers. Little's Law gives in-flight = {{c5::throughput x latency}}.