A parade where everyone maintains exact position. The drummer at position 10 stays at position 10. The flag bearer at position 50 remains at position 50. Even if they take breaks, when they reassemble, order is preserved.
Kafka ordering guarantees messages march in formation, maintaining sequence across a distributed network.
Without Ordering
Town festival without organization:
- Mayor starts at front, ends in middle
- Marching band scatters throughout
- Veterans’ group loses members
- Floats pass each other randomly
Spectators confused. Not a parade, a mob.
Multiple Routes
Three parade routes (Main Street, Second Avenue, Third Boulevard). Participants choose randomly. Now three different parades, each telling a different story.
Spectator on Main Street sees veterans first. Second Avenue starts with school band. The narrative is lost.
Kafka’s Approach
Partitions as Lanes
This diagram requires JavaScript.
Enable JavaScript in your browser to use this feature.
Each lane (partition) maintains perfect order.
Position Tracking
Every participant gets a permanent position:
- Partition 0, Position 0: First veteran
- Partition 0, Position 1: Second veteran
- Partition 0, Position 2: Third veteran
Position is immutable. Even if veteran #1 takes a break, returning means returning to position 1.
Ordering Guarantees
Within a Partition: Absolute Order
Messages A, B, C sent to Partition 0. They always process as A, B, C. No exceptions, no reordering.
Across Partitions: No Guarantees
Partition 0: A, B, C Partition 1: X, Y, Z
Processing order could be A, X, B, Y, C, Z or X, Y, Z, A, B, C. Lanes march independently.
Key Decision
Choose how to assign messages to partitions:
- By type: All veterans in one partition
- By geography: North-side together
- By time: Morning arrivals in Partition 1
- Random: Load balance
Choose based on what ordering matters for.
Offset Management
Each position has an offset (its number in line):
Partition 0:
- Offset 0: Flag bearer enters
- Offset 1: First veteran enters
- Offset 2: Second veteran enters
- Offset 1000: Final veteran enters
Consumers track their position: “I’ve processed up to offset 456.”
Append-Only
New participants only join at the end:
Current parade end: Offset 1000 New participant: Gets offset 1001 No insertion in middle, no reordering, only forward progress.
Common Problems
Partition Skew
One lane gets all participants:
Lane 0: 10,000 marchers Lane 1: 100 marchers Lane 2: 50 marchers
Causes: Poor key selection, temporal patterns, celebrity effect.
Solutions: Better key distribution, null key randomization, partition reassignment.
Slow Consumer
Viewing stand can’t keep up:
Parade at position 10,000 Stand A still at position 100 Massive lag, risk of timeout.
Solutions: Scale viewing stands, skip non-essential participants, batch processing.
Rebalancing
Consumer stands redistribute work:
Stand A watching partitions 0,1,2 Stand B joins Rebalance: A gets 0,1 and B gets 2 Brief pause during handoff, then continue.
Decision Rules
Kafka guarantees order within partitions, not across them.
Use partitioning keys when:
- Related messages must stay ordered
- You can identify a good partitioning key
Don’t expect global order. Design for order within logical boundaries.
The next time you build a streaming system, think about what formations must be preserved and where marchers can proceed independently.