Sticky Assignment: Why Rebalances Got So Much Cheaper

Every time a consumer joins or leaves a group, the group coordinator runs the assignor and ships a new assignment to every member. With the classic RangeAssignor or RoundRobinAssignor, the new plan is computed from scratch — there's no awareness of what each consumer was already doing. That meant every rebalance was a stop-the-world event: every consumer revoked every partition, re-joined the group, and got a brand-new assignment, even if the work was almost identical to before.

StickyAssignor (and its successor CooperativeStickyAssignor) fix this.

What "Sticky" Means

When a sticky assignor runs, it has two goals:

1. Balance — partitions should be distributed as evenly as the partition count allows.
2. Stickiness — each consumer should keep as many of its existing partitions as possible.

When goal 1 and 2 conflict (one consumer is over-assigned), the assignor moves the minimum number of partitions to restore balance.

The payoff is huge for stateful consumers. A Kafka Streams app that joins a topic with a local RocksDB store doesn't have to rehydrate from a changelog every time a different consumer leaves the group.

Range vs Round-Robin vs Sticky

For 6 partitions, 3 consumers A, B, C:

The sticky case in column 3 is the obvious "minimum-disruption" plan: A and B don't move, the new consumer D inherits exactly what C had. Range and round-robin reshuffle every member.

Cooperative (Incremental) Rebalancing

Sticky's first version still triggered a stop-the-world revoke/assign cycle: every consumer revoked all its partitions, the assignor ran, and consumers re-acquired (mostly the same) partitions. So even when nothing logically moved, processing paused.

CooperativeStickyAssignor (KIP-429) goes further: it does the rebalance in two phases.

1. The coordinator hands each consumer the new assignment but only asks them to revoke the partitions they are about to lose. Consumers continue processing the partitions they're keeping.
2. After all revocations confirm, the coordinator runs a second short round to assign the freed partitions to their new owners.

Net effect: a consumer that keeps 9 of its 10 partitions stays fully online for 9 of them throughout the rebalance. This is the default for new Kafka Streams apps and is the recommended choice for any new consumer group.

When You Might Pick Something Else

• RangeAssignor is still the historical default for consumer (not Streams) apps, but mostly because it's been there forever.
• RoundRobinAssignor is occasionally chosen when the group's members subscribe to different topic sets and you want round-robin within each topic.
• CooperativeStickyAssignor is the default for any new design. The only reason to skip it is interop with very old clients that don't understand the cooperative protocol.

What This Lesson Asks

The exercise is to implement the sticky assignor's single goal: given a current assignment and a new member set, compute the new assignment that keeps stickiness while restoring balance. You'll see why production rebalances now reshuffle a handful of partitions instead of every single one.