🔢 Partitions, consumer groups and ordering limits

Learning Objectives

By the end of this lesson you can:

  • State precisely what Kafka and RabbitMQ each guarantee about message order, and where that guarantee stops.
  • Explain how a partition key maps to a partition, and why changing the partition count remaps keys.
  • Describe how a consumer group assigns partitions, and why concurrency is the usual cause of reordering.
  • Choose a keying strategy (keyed, unkeyed, single partition, sharded queue) for a stated ordering requirement.

Quick Review

We finished the delivery-semantics module with exactly-once: Kafka's idempotent producer absorbs retries, transactions commit output and offsets atomically, and read_committed hides aborted records. RabbitMQ reaches at-least-once plus idempotent consumers. One phrase from that lesson does all the work today: Kafka's transaction scope is per partition. Today we find out why that phrase exists, and what RabbitMQ offers in its place.

Theory

Kafka: the partition is the unit of order

A Kafka topic is not one log. It is N partitions, each an append-only log with its own monotonically increasing offsets. Producers append at the end, consumers read in offset order. So:

==Kafka orders messages within a single partition, and nowhere else.==

Two records in different partitions have no defined order, even if you produced them milliseconds apart. This is not a limitation to work around; it is the design that lets you scale writes horizontally.

How a key becomes a partition. The default partitioner hashes the message key with {{c1::murmur2}} and takes the result modulo the partition count. Same key, same partition count, same partition: always. That is the whole keying contract. Null keys take a different path: the sticky partitioner sends a batch to one partition, then rotates. Good for throughput, useless for ordering.

Consumer groups. Consumers in a group divide the partitions between them, and each partition is assigned to {{c2::exactly one consumer}} at a time.

flowchart LR
  subgraph topic["🎯 Topic orders, 3 partitions"]
    P0["📜 Partition 0"]
    P1["📜 Partition 1"]
    P2["📜 Partition 2"]
  end
  subgraph group["👥 Consumer group billing"]
    C1["Consumer A"]
    C2["Consumer B"]
  end
  P0 --> C1
  P1 --> C1
  P2 --> C2

Three consequences follow. First, parallelism inside a group is capped by partition count: a fourth consumer sits idle. Second, per-partition order survives because exactly one consumer reads each partition sequentially. Third, the moment you hand records to a thread pool inside your consumer, you have reordering again, even though Kafka delivered them in order.

Two classic ways to lose Kafka ordering. Turning the partition count from 6 to 8 is the big one: changing the number of partitions {{c5::remaps keys to different partitions}}, so events for the same entity can straddle two partitions during the change. And setting max.in.flight.requests.per.connection above 1 without idempotence lets a retried batch land after a newer one.

RabbitMQ: the queue plus one consumer is the unit of order

In RabbitMQ the queue is the ordered structure, and the broker delivers in FIFO order to whoever is bound to it. When several consumers share a queue, the broker round-robins: message 1 to consumer A, message 2 to consumer B, and so on. Each message is processed once, but nothing ties completion order to enqueue order.

So RabbitMQ guarantees FIFO only when that queue is consumed by {{c3::a single consumer}}. Widen that and you get more throughput without ordering. The escape hatches are:

  • x-single-active-consumer, which gives a queue {{c4::one active consumer at a time}} with standby failover, preserving order.
  • Sharding: N queues, each with one consumer, with an exchange (for example the consistent hash exchange) routing by entity id. This recreates Kafka's per-key ordering with a queue per shard.

Also remember that a nack with requeue, or a consumer crash, puts a message back near its original position while later messages may already have reached your downstream system. Ordering is preserved in the queue, not in your side effects.

Worked Examples

Example 1: keying a payment stream

You publish account events for 400,000 accounts into a topic with 6 partitions. Requirements: debits must be applied before the matching credit.

Step 1. Key by accountId. Account 42 always hashes to the same partition. Step 2. All 400,000 keys spread across 6 partitions, so consumers stay busy. Step 3. A debit and its credit for account 42 land in one partition, in production order, and one consumer reads them in that order.

Now change the key to country. Two events for account 42 can land on different partitions and be consumed by different consumers at different speeds. The credit can be applied first. The key is the ordering boundary; if you key by something coarser than your ordering unit, you have no ordering for that unit.

Example 2: three consumers on one RabbitMQ queue

An order queue has three consumers. Message 101 (create order) goes to consumer A, message 102 (cancel order) goes to consumer B. B is faster, so your downstream service sees the cancel before the create. Nothing is lost, nothing is duplicated, and the state is still wrong.

Fixes, in order of effort:

  1. Set prefetch to 1 and run one consumer for that queue. Throughput for this queue is now one message at a time.
  2. Split into 8 sharded queues keyed by orderId and run one consumer per shard. Ordering per order, parallelism across orders.
  3. Enable x-single-active-consumer if you mostly want failover, not throughput.

Example 3: repartitioning an existing topic

Your users topic has 6 partitions, keyed by userId. Traffic doubles, so you bump to 12.

Step 1. Before the change, userId=17 maps to partition 2. Step 2. After the change, the same key can map to partition 5. The hash is unchanged; the modulus is not. Step 3. Events for user 17 now exist in both partitions, read by different consumers. Order for user 17 is broken for the overlap window.

Rules of thumb: over-provision partitions at creation, or treat a partition-count change as a migration with a drain period. For a hot key, add a composite key like US#accountId rather than more partitions.

Discussion and Reflection

  • Your team wants to add a fourth consumer to a group sitting on a 3-partition topic. What do you tell them, and what do you propose instead?
  • Which of your current systems relies on ordering that was never actually guaranteed by the broker?
  • If a downstream effect is idempotent, does ordering still matter? Where exactly?

Expert Corner

  • Write the ordering unit down. One sentence: "we need order per X." Then check that the key equals X exactly. Most reordering incidents are a mismatch between that sentence and the producer's key expression.
  • Low-cardinality keys are hot partitions. Keying by country, tenant tier, or status funnels traffic to a few partitions while the rest idle.
  • Unkeyed topics are ordering-free by design. If anyone assumes order on a null-key topic, that assumption is already broken.
  • RabbitMQ: one queue, one consumer, prefetch 1 is the only configuration that gives strict FIFO without extra machinery. Everything else is a deliberate trade.
  • Rebalances pause, they do not reorder. Partition reassignment keeps per-partition order; the risk is your own in-consumer concurrency.
Kafka guarantees order {{c1::within a single partition}} and provides no ordering across partitions.
RabbitMQ guarantees FIFO order only when a queue is consumed by {{c2::a single consumer}}.
Increasing a Kafka topic's partition count {{c3::remaps existing keys}} and breaks key-level ordering.

Review Deck

[
  { "front": "What is the unit of ordering in Kafka?", "back": "A single partition. There is no ordering guarantee across partitions." },
  { "front": "What is the unit of ordering in RabbitMQ?", "back": "A single queue consumed by a single consumer." },
  { "front": "How does Kafka's default partitioner map a key?", "back": "murmur2 hash of the key, modulo the partition count, so the key stays on one partition while the count is unchanged." },
  { "front": "What happens to key-to-partition mapping when you add partitions?", "back": "Existing keys can move to different partitions, so key-level ordering breaks during the change." },
  { "front": "What does a null key mean for Kafka ordering?", "back": "The sticky partitioner spreads batches across partitions, so there is no per-key ordering." },
  { "front": "In a consumer group, can two consumers read the same partition?", "back": "No. Each partition is assigned to exactly one consumer at a time." },
  { "front": "What caps parallelism inside a Kafka consumer group?", "back": "The partition count. Extra consumers beyond it stay idle." },
  { "front": "Why do competing consumers on one RabbitMQ queue reorder messages?", "back": "The broker round-robins deliveries, so whichever consumer finishes first sets the downstream order." },
  { "front": "Which RabbitMQ feature keeps order while allowing failover?", "back": "x-single-active-consumer: one active consumer per queue, with standby consumers taking over." },
  { "front": "What does a nack with requeue do to ordering?", "back": "The message returns near its original position while later messages may already be processed downstream." },
  { "front": "How do you get per-entity ordering in RabbitMQ beyond one consumer?", "back": "Shard into N queues (for example with a consistent hash exchange) and run one consumer per queue." },
  { "front": "What causes a hot Kafka partition?", "back": "A low-cardinality key, such as country, sending most traffic to one partition." },
  { "front": "Which producer setting can reorder retried batches?", "back": "max.in.flight.requests.per.connection greater than 1 without idempotence enabled." }
]

Summary and Next Steps

Ordering is a property of a container, not of a broker: a Kafka partition, or a RabbitMQ queue with a single consumer. Your key is the choice that decides which container an event lands in, so the key must equal the unit you need ordered, and it must not change shape underneath you. Consumer groups give you parallelism by splitting containers, which is exactly why adding consumers never adds ordering.

Next lesson: throughput, latency and backpressure compared. We take these same containers and ask how much work they can absorb per second, what happens when producers outrun consumers, and which broker pushes back on you versus letting you fall behind.