Throughput, latency and backpressure compared

Learning objectives

• Explain how batching, compression and sequential disk writes give Kafka its throughput ceiling.
• Explain how prefetch and per-message acknowledgements shape RabbitMQ's latency floor and in-flight limit.
• Size prefetch and consumer concurrency with Little's Law instead of guessing.
• Predict which broker saturates first for a given volume and latency target, and say why.

Quick review

Last lesson: ordering is a property of a container, not a broker. A Kafka partition orders; a RabbitMQ queue with one consumer orders. The key decides which container an event lands in.

Today we reuse that idea from the other side. The number of containers is also your parallelism budget. More partitions means more parallel batches. More queues means more parallel consumers. That single sentence explains most of the throughput difference you will measure.

Theory

Kafka: throughput comes from amortization

Kafka writes to an append-only log. Writes are sequential, and sequential I/O on a single spinning disk beats random I/O by orders of magnitude. Reads come from the page cache and go out through zero-copy sendfile. Neither path needs a per-message index lookup, which is why read cost stays flat as the log grows to terabytes.

The producer then amortizes everything else over a batch:

• batch.size is the byte ceiling per partition.
• linger.ms is the time ceiling. A record waits up to this long for company.
• Compression (lz4, zstd) is applied per batch, so bigger batches compress better.

The trade is explicit: linger.ms is latency you pay on every single record. On the consumer side, fetch.min.bytes and fetch.max.wait.ms do the same job in reverse.

RabbitMQ: latency comes from push and small units of work

RabbitMQ pushes a message to a consumer, waits for an ack, and repeats. Every message is routed, delivered, tracked and acknowledged individually. That per-message bookkeeping is expensive at volume, but it is also why a single message can cross the broker in well under a millisecond when the queue is empty.

You cannot batch your way out of this. The lever is basic.qos prefetch: how many unacknowledged messages a consumer may hold. Prefetch 1 gives strict serialization and the smallest redelivery window. Prefetch 250 parks 250 messages in one consumer's memory and, if that consumer dies, redelivers all 250. Quorum queues add a Raft majority round trip to every publish and ack.

Backpressure: pull versus push

This is the part that surprises people in production.

RabbitMQ applies backpressure explicitly. A slow consumer grows the queue, queue depth eats memory, the memory high watermark (default 0.6 of RAM) or disk_free_limit trips, and the broker blocks publishers. The client sees connection.blocked. Your producers slow down, whether you planned for it or not. It is crude but honest: nothing is silently lost.

Kafka consumers pull. A slow consumer cannot push back on the broker, because the broker never pushes. The consumer simply accumulates lag, and the log absorbs the difference until retention.ms or retention.bytes deletes it. So ==Kafka's backpressure signal is data expiring before it is read==, which is far quieter than a blocked connection. Producer-side backpressure is separate: buffer.memory fills and sends block for up to max.block.ms.

Worked examples

Example 1: sizing prefetch with Little's Law

In-flight messages equal throughput multiplied by latency: $L = \lambda \times W$.

You need 6,000 msg/s across 24 consumers, so 250 msg/s per consumer. p99 processing time is 40 ms.

$$ L = 250 \times 0.040 = 10 $$

Prefetch must be at least 10, and you need roughly 10 concurrent workers per consumer to feed them. Compare the naive setup: prefetch 250 with one thread gives $1/0.040 = 25$ msg/s no matter what. You would be 10x short while holding 250 unacked messages as a redelivery bomb.

Example 2: what linger.ms actually buys you

Producer volume is 15,000 messages/s at 1 KB, so 15 MB/s. With 6 partitions, each partition sees 2.5 MB/s. batch.size is 1 MB, linger.ms is 5.

Filling 1 MB at 2.5 MB/s takes 400 ms, so the size ceiling never fires. The actual batch is $2.5 \text{ MB/s} \times 0.005\text{ s} = 12.5$ KB, about 12 records.

You are paying per-batch overhead 200 times a second. Raise linger.ms to 20 if your latency budget allows: batches grow to 50 KB, overhead drops 4x, and lz4 starts finding real redundancy. batch.size is a ceiling, linger.ms is the dial at moderate volume.

Example 3: which one saturates first

Workload: 40,000 events/s at 2 KB, so 80 MB/s, with a p99 end-to-end target of 200 ms.

Kafka: spread over 12 partitions, each writes 6.7 MB/s sequentially, comfortable for one node. Latency is linger.ms plus one acks=all round trip plus fetch, so tens of milliseconds. Headroom is real, and if you lose it you add partitions and brokers.

RabbitMQ: 40,000 msg/s with per-message acks on quorum queues needs many queues and connections, and every publish pays a majority round trip. Any consumer stall grows the queue, raises memory, and triggers flow control that blocks your publishers. p99 degrades long before the mean does.

Verdict: sustained 80 MB/s with a 200 ms budget belongs on Kafka. Flip the numbers, 200 events/s with a 5 ms budget, and RabbitMQ wins easily because nothing waits for a batch.

[!NOTE] These figures are order-of-magnitude reasoning, not benchmarks. Measure your own message size, replication factor and hardware before committing.

Discussion & reflection prompts

1. Your RabbitMQ publishers got connection.blocked during a consumer deploy. What did the broker protect you from, and what did it hide?
2. Your Kafka consumer group is 6 hours behind and retention is 24 hours. What alarm would you rather have than lag, and why is it hard to build?
3. Where in your system would a 20 ms linger.ms be invisible, and where would it break an SLA?

Expert corner

• Benchmark with realistic depth. An empty RabbitMQ queue answers in microseconds. Once depth grows into the millions, a single queue's Erlang process becomes the bottleneck, while Kafka's read cost does not move.
• Report p99, never the mean. Batching and backpressure both live in the tail.
• Set linger.ms to roughly a tenth of your latency budget. Below that you are buying throughput you already have; above it you are burning SLA.
• Treat prefetch as a memory and risk budget, not a speed dial. Throughput comes from concurrency and consumer count.
• acks=all and quorum queues both add a replication round trip. Budget for it in the latency target rather than discovering it in production.
• Watch two different signals: consumer lag for Kafka, queue depth plus unacked count for RabbitMQ. They fail in opposite directions.

Summary & next steps

Kafka buys throughput by amortizing cost over batches and sequential writes, and pays for it in a latency floor set by linger.ms. RabbitMQ buys low latency with per-message push and ack, and pays for it in per-message overhead and a single-queue ceiling. Backpressure differs just as sharply: RabbitMQ throttles the producer, Kafka lets the consumer fall behind into the retention window.

Next up: routing and topology. You will see how exchanges and bindings compare with topics and partitions, and how routing complexity changes the throughput numbers we just built.

[
  { "front": "What gives Kafka its high throughput?", "back": "Sequential log appends, plus batching and compression that amortize cost over a whole batch instead of one record." },
  { "front": "Which two producer settings control Kafka batching?", "back": "batch.size (byte ceiling per partition) and linger.ms (time ceiling)." },
  { "front": "What does linger.ms cost you?", "back": "Latency: every record waits up to that long for a batch to fill." },
  { "front": "What controls in-flight messages in RabbitMQ?", "back": "Prefetch (basic.qos): the maximum unacknowledged messages a consumer may hold." },
  { "front": "High prefetch with a single-threaded consumer: what happens?", "back": "Throughput stays at 1/processing time, but many messages sit unacked and all get redelivered if the consumer dies." },
  { "front": "What is Kafka's backpressure signal on the consumer side?", "back": "Consumer lag. The log absorbs it silently until retention expires and the data is deleted." },
  { "front": "How does RabbitMQ push back on producers?", "back": "Memory high watermark or disk_free_limit trips flow control and blocks publishers (connection.blocked)." },
  { "front": "Little's Law as used for prefetch sizing?", "back": "In-flight = throughput x latency (L = lambda x W)." },
  { "front": "Why does Kafka read cost stay flat as the log grows?", "back": "Consumers fetch sequentially by offset; there is no per-message index scan or deletion." },
  { "front": "Which settings add a replication round trip to publish latency?", "back": "acks=all with min.insync.replicas in Kafka, and publisher confirms on RabbitMQ quorum queues." },
  { "front": "When does RabbitMQ beat Kafka on latency?", "back": "Low to moderate volume with small messages, when per-message delay dominates and you cannot afford a batching wait." },
  { "front": "What does fetch.max.wait.ms do?", "back": "Caps how long a Kafka consumer waits to reach fetch.min.bytes before returning a smaller batch." }
]

Cloze check: Kafka amortizes cost per {{c1::batch}} rather than per message. RabbitMQ's prefetch caps {{c2::in-flight unacknowledged messages}} per consumer. Kafka consumers are {{c3::pull}}-based, so a slow consumer shows up as {{c4::consumer lag}} instead of throttling producers. Little's Law gives in-flight = {{c5::throughput x latency}}.