Sticky Assignment

After this lesson your leader recomputes a group's assignment after a join or a crash while moving as few partitions as it can, and it reports how many partitions had to change owner.

Why moving partitions is expensive

When a partition changes owner, the new owner has to fetch the committed offset, rebuild whatever local state belongs to that partition (a cache, a RocksDB store in Kafka Streams), and warm up. A range or round-robin assignor recomputes from scratch, so a single member joining can reshuffle most of the group. The sticky assignor has two goals, in this order:

1. Balance: no member owns more than one partition more than another.
2. Stickiness: subject to balance, keep every partition with its previous owner.

Where does the leader learn the previous owners? Each member's subscription (the metadata from the previous lesson) carries its currently owned partitions from version 1 on. The exercise gives you them directly.

The algorithm, as a recipe

Everything that is decided must be decided the same way by any implementation, because the leader's answer is all the group ever sees. So the recipe pins down each choice.

Let P be the number of partitions and M the number of members now in the group.

1. Targets. Each member's target is floor(P / M) partitions, and P mod M members get one more. The ones that get it are the members that already own the most (ties go to the smaller id). That choice keeps the most partitions in place.
2. Orphans. Partitions whose previous owner is not in the group any more are unassigned.
3. Shed. A member that owns more than its target gives up its highest partition ids until it is at its target. They join the unassigned pool.
4. Fill. Take the unassigned partitions in ascending order. Give each to the member that is still below its target and currently owns the fewest; ties go to the smaller id.
5. Report how many partitions ended with a different owner than before (orphans count, shed partitions count).

Why fill in ascending order, and why shed the highest ids? Not because it matters for balance, but so that two correct implementations produce the same output. Determinism is part of the protocol.

Worked trace

Different data from your exercise: members X, Y, Z (a member W crashed, Z is new). W owned 0,1,2, X owned 3,4,5,6, Y owned 7,8.

P = 9, M = 3  ->  floor 3, remainder 0: every target is 3
current:  X=[3,4,5,6]  Y=[7,8]  Z=[]
orphans (W left): 0,1,2
shed:     X has 4 > 3, gives up its highest id: 6       pool = [0,1,2,6]

fill 0 -> Z (owns 0, fewest)                              Z=[0]
fill 1 -> Z (owns 1, Y owns 2)                            Z=[0,1]
fill 2 -> Y and Z both own 2, smaller id wins: Y          Y=[2,7,8]
fill 6 -> Z (the only one below target)                   Z=[0,1,6]

X: 3,4,5   Y: 2,7,8   Z: 0,1,6      moved = 4 (0,1,2 and 6)

Cooperative rebalancing

The eager protocol in this lesson revokes every partition from every member before reassigning. The cooperative sticky assignor (KIP-429) runs the same computation but in two rounds: members first give up only the partitions that must move, and everyone keeps processing the rest. That is why CooperativeStickyAssignor is the default for Kafka Streams since 2.4: the stop-the-world pause of an eager rebalance disappears for the partitions that do not move.

Common mistakes

• Fixing the extras by id. Giving the +1 partitions to the smallest ids moves a partition off the member that has the most. The extras go to the heaviest members.
• Releasing the lowest ids. The recipe sheds the highest first; do the same or the output differs.
• Forgetting orphans. Partitions of a member that left must be reassigned too.
• Filling to equal counts instead of targets. A member that is at its target must not receive more, even if it ties for fewest.
• Counting moves wrongly. A partition kept by its previous owner is not moved; everything else is.

Your exercise

The starter parses the input and prints the answer. Implement the rebalance. The most likely mistake is the order of the shed and fill steps, so trace the worked example by hand first.