Read

Sticky Assignment

Every time a consumer group rebalances — a consumer joins, leaves, or crashes — the group coordinator has to decide which consumer owns which partitions. The naive approach (RangeAssignor or RoundRobinAssignor) just recomputes an assignment from scratch based on the new membership. That's simple, but it's expensive in practice: every partition that moves to a new owner forces that consumer to flush any in-flight state, commit offsets, and re-fetch data it may have just fetched. For stateful consumers (e.g. Kafka Streams tasks with local RocksDB stores), a partition moving means throwing away and rebuilding that store.

The StickyAssignor (and its cooperative successor, CooperativeStickyAssignor) optimizes for a different goal: minimize the number of partitions that change owners, while still keeping the assignment balanced.

Two competing constraints

1. Balance — no consumer should have more than one extra partition compared to any other. With P partitions and M consumers, every consumer should end up with either floor(P/M) or ceil(P/M) partitions.
2. Stickiness — a consumer that already owns a partition should keep it, unless keeping it would violate balance.

These pull against each other: if you only cared about balance, you could reshuffle freely. If you only cared about stickiness, you might leave the group badly skewed (e.g. a consumer that left the group leaves its partitions completely unclaimed).

The algorithm shape

A practical way to implement this:

1. Seed the new assignment from history. For every consumer still in the group, start with the partitions they previously owned.
2. Collect orphans. Any partition owned by a consumer that has since left the group becomes "unassigned" and needs a new home.
3. Compute target counts. With P partitions and M members, floor(P/M) members get the floor count and the remaining P % M get one extra (ceil). Decide deterministically which members get the "+1" (e.g., sort ascending and give the extra to the first P % M).
4. Trim over-full consumers. If preserving history left some consumer with more than its target, move its excess partitions into the unassigned pool. Which partitions it gives up has to be pinned down, not left to taste: shed the highest partition ids first, so a consumer holding [0,1,2,3] and targeted at 2 keeps 0,1 and releases 3 then 2. Every member computes the assignment locally, so an unstated tie-break is a bug, not a detail.
5. Fill under-full consumers. Repeatedly hand out unassigned partitions to whichever consumer currently has the fewest, breaking ties alphabetically by consumer id, until every consumer reaches its target and the unassigned pool is empty.

Why determinism matters

Every consumer in the group runs the assignment strategy locally against the same input (the group coordinator ships the full membership and ownership metadata to the elected group leader, who computes assignments for everyone). If two implementations could produce different results from the same input, consumers would disagree about ownership. That's why the tie-breaking rules (lexicographic consumer id, "give the extra partition to the smallest id first") aren't arbitrary — they have to be specified precisely enough that any correct implementation converges on the same answer.

Trace through a small example

Members A, B, D (C left the group). A owned [0,1], B owned [2,3], C owned [4,5]. D is new, with nothing. Six partitions over three members means every consumer should end up with exactly 2. A and B are already at their target, so they keep their partitions untouched. C's partitions 4,5 are orphaned and go to D, the only consumer below target. Nothing moved that didn't have to.