Routing and topology: exchanges, bindings and topics

Learning Objectives

By the end of this lesson you can:

• Describe how RabbitMQ resolves a message to a set of queues using exchanges, bindings and routing keys.
• Explain the * and # wildcard rules in a topic exchange, and predict what a given binding key will match.
• Reproduce the same business notification flow on Kafka using topics, partition keys and consumer groups.
• Name the concern that moves from the broker to the application when you switch from RabbitMQ topology to Kafka topology.
• State, for a given change request (new channel, new event type, new filter), which topology absorbs it cheaply and which one does not.

Quick Review

From the last two lessons: inside a Kafka consumer group, partitions are split across members, so each message is processed by exactly one member of that group. Ordering is per partition. Throughput comes from batching and sequential writes, and linger.ms sets your latency floor.

Both facts matter here, because a Kafka consumer group is the closest thing Kafka has to a routing primitive. Hold that thought.

Theory

RabbitMQ: routing is a graph the broker executes

A producer never publishes to a queue. It publishes to an exchange with a routing key, and the broker consults its bindings to decide which queues get a copy. ==RabbitMQ routes on the broker, Kafka routes in the producer and the consumer group.==

Four exchange types do the work:

Topic exchange wildcards are the part people get wrong in review:

• * matches exactly one word.
• # matches zero or more words.
• Words are separated by dots.

So order.* matches order.created and order.paid, but not order.created.eu. order.# matches all three, and also bare order.

The topology itself (exchanges, bindings, dead-letter exchanges, alternate exchanges) is broker-side state you version and review like code. That is real operational cost, and it buys you consumers that receive only what they asked for.

Kafka: routing is naming plus group identity

Kafka has no bindings. A topic is a partitioned log, the producer's partitioner picks the partition (key hash, or round-robin with no key), and every consumer group reads the whole log independently. Fan-out to five services costs five reads and one stored copy.

There is no server-side filter. If a consumer wants only ORDER_CREATED, it either reads everything and drops the rest in code, or somebody creates a narrower topic upstream (a producer-side router, or a Kafka Streams job). That is the whole routing model.

Where the complexity moves

In a RabbitMQ topic exchange, `*` matches {{c1::exactly one word}} while `#` matches {{c2::zero or more words}}.
A Kafka topic fans out to every {{c3::consumer group}}, but parallelism inside one group is capped by {{c4::the partition count}}.
Adding a new channel in Kafka costs {{c5::no extra stored copy of the data}}, at the price of that consumer reading the {{c6::entire topic}} unless the data was split upstream.

Worked Examples

Example 1: order notifications on RabbitMQ

Business flow: when an order is created, paid or cancelled, email the customer, SMS them on creation only, run fraud checks on everything, and feed analytics with all events.

1. Declare one topic exchange named `notifications`.
2. Declare queue `email.order` and bind it to `notifications` with key `order.*`.
3. Declare queue `sms.order` and bind it with key `order.created`.
4. Declare queue `fraud.check` and bind it with key `order.#`.
5. Declare queue `analytics.all` and bind it with key `#`.
6. Publish with routing key `order.paid`. The broker copies the message to `email.order`, `fraud.check` and `analytics.all`, and not to `sms.order`.

Four queues, four bindings, zero routing logic in any consumer. A new WhatsApp channel is a new queue plus one binding, and the order service publisher is not touched.

Example 2: the same flow on Kafka

topic: orders.events   partitions: 6   key: orderId

consumer group: email-service       -> all order events
consumer group: sms-service         -> filters eventType == ORDER_CREATED
consumer group: fraud-service       -> all order events
consumer group: analytics-loader    -> all order events

Fan-out is automatic: four groups, four full reads, one stored copy. Keying by orderId means event 42's email always arrives after event 42's payment, which is the ordering guarantee you actually care about. The cost you just accepted is that sms-service reads the entire topic to find its subset.

Example 3: an EU-only compliance consumer

New requirement: a consumer needs only EU order events.

• RabbitMQ: bind a queue with order.#.eu. Done. Cost is one more message copy on the matching paths.
• Kafka: there is no binding. Either the consumer reads all partitions and discards non-EU events (simple, but the read bandwidth is already spent), or you add a topic orders.eu populated by a router, which now needs its own ownership, retention policy and schema review.

This is the decision pattern to remember: RabbitMQ moves routing cost to topology time, Kafka moves it to read time and topic governance.

Example 4: when the wildcard bites

A teammate binds order.* expecting to catch a retry event published as order.created.retry. It never arrives: three words, one wildcard. They needed order.*.* or order.#. In Kafka the equivalent bug does not exist, and the equivalent bug is a startsWith filter that quietly lets a variant event through. Different failure mode, same lesson: write the routing contract down.

Discussion & Reflection

1. Your team has 12 downstream services and 4 event types. Sketch both topologies and count the artifacts you would have to own and version in each.
2. Somebody proposes # bindings "so we never miss anything". What does that do to the broker's copy count and to your ability to reason about who consumes what?
3. In the Kafka design above, sms-service reads 100 percent of the topic to process 20 percent of messages. At what volume ratio would you push for a dedicated topic, and who pays for that decision?

Expert Corner

• Rule of thumb: if the routing rule is stable and small, RabbitMQ bindings repay their setup cost in every consumer you never had to write filter code for. If the routing rule changes weekly, bindings become config drift.
• Common mistake: treating a Kafka consumer group as a filter. It is a scaling and fan-out unit, not a predicate.
• Common mistake: assuming RabbitMQ fan-out preserves per-entity ordering once a queue has multiple competing consumers. It does not; that is the same partition problem Kafka names explicitly.
• Watch the copy economics: on RabbitMQ, a message bound to eight queues costs eight copies. That is fine at notification volume and painful at telemetry volume. Kafka stores one and lets the groups fight over read bandwidth.
• Write the routing contract: exchange, key patterns and queue names for RabbitMQ; topic, key and event types for Kafka. Review changes to it like an API change.

Summary & Next Steps

RabbitMQ gives you a broker-side routing graph: exchanges, bindings, wildcard keys, and consumers that only see their slice. Kafka gives you a log plus group identity: fan-out for free, filtering on the client, ordering per partition key. Your job as the engineer is to say which of those costs you would rather carry for this particular flow.

You can now map a notification flow onto both and name where each topology adds complexity. The next lesson is the module checkpoint, where you will be asked to predict ordering and routing behavior under load and justify a broker choice for a described scenario.

[
  { "front": "In RabbitMQ, what does a producer publish to?", "back": "An exchange, with a routing key. It never publishes straight to a queue, except via the default exchange." },
  { "front": "What is a RabbitMQ binding?", "back": "A broker-side rule on an exchange stating which queues receive messages matching a routing key pattern." },
  { "front": "RabbitMQ topic exchange: what does `*` match?", "back": "Exactly one dot-separated word." },
  { "front": "RabbitMQ topic exchange: what does `#` match?", "back": "Zero or more dot-separated words." },
  { "front": "Which exchange type ignores the routing key and copies to every bound queue?", "back": "fanout" },
  { "front": "Which exchange type routes on message header attributes?", "back": "headers" },
  { "front": "How does the RabbitMQ default exchange route?", "back": "By queue name used as the routing key, with no binding declared." },
  { "front": "How does Kafka deliver one event to several independent services?", "back": "Each service uses its own consumer group, and every group reads the full topic." },
  { "front": "What is Kafka's equivalent of a RabbitMQ binding with a filter?", "back": "None on the broker: the consumer filters client-side, or a narrower topic is created upstream." },
  { "front": "What decides the Kafka partition a message lands in?", "back": "The producer's partitioner: hash of the key, or round-robin when no key is set." },
  { "front": "What is Kafka's ordering scope?", "back": "Per partition, so keying by an entity ID keeps that entity's events in order." },
  { "front": "Cost of a fifth consumer group in Kafka versus a fifth queue in RabbitMQ?", "back": "Kafka: no extra stored copy, the log is simply read again. RabbitMQ: one more copy of every matching message." },
  { "front": "What does adding a RabbitMQ binding require from producers?", "back": "Nothing. Bindings are broker-side topology state, so no producer redeploy is needed." },
  { "front": "Why does the binding `order.#.eu` catch `order.created.eu` but `order.*.eu` not?", "back": "`#` covers zero or more words, `*` covers exactly one." }
]