Slow Start: Ramping New Backends

A backend that has just started is functionally slower than its steady-state self. Its application caches are empty, its database connections are unopened, its JIT has not warmed, its CPU caches are cold. Hand it a full share of traffic in its first second and those first requests run slow, and now your own protection turns on you. Timeouts fire, passive checks count errors, the circuit breaker trips, and a perfectly good server is ejected, re-added, still cold, and ejected again. This oscillation is called flapping, and it can keep a healthy machine out of rotation indefinitely.

Slow start fixes it by ramping a backend's effective weight linearly from 0 up to its configured target over a window.

The ramp

effective(t) = target * min(elapsed, ramp) / ramp          elapsed = t - joined_at, never below 0

All integers, with the division last and rounding down. A backend with target 10 and a 20 second ramp:

t - joined   0    5    10   15   19   20   100
effective    0    2    5    7    9    10   10
                  ^ 10*5/20 = 2.5, truncated to 2

At the moment of joining the weight is 0, so the backend receives no traffic at all, not "a little". When the window is over the weight is the target and stays there. A ramp of 0 means no ramp: the weight is the target immediately, which is what every backend in the initial pool gets.

Every serious balancer ships this: NGINX Plus (slow_start=30s on an upstream server), HAProxy (slowstart 30s), Envoy (slow_start_config, with an aggression curve) and AWS ALB (slow start on target groups, 30 to 900 seconds, off by default). The second trigger matters as much as the first: slow start applies to backends recovering from DOWN as well. A server that has just passed its health checks has exactly the cold-cache problem of a fresh one, and blasting it on recovery is how you kill it again.

Plugging into smooth weighted round-robin

The pleasant part is that the selection algorithm does not change. You still run the smooth weighted round-robin from the weighted lesson, feeding it each backend's current effective weight in place of the fixed one:

1. Take the backends that are UP and have an effective weight above 0. Only they take part.
2. Add each participant's effective weight to its current score.
3. The highest current wins; ties go to the backend listed first in the pool.
4. Subtract the total of the participating weights from the winner.

Two details bite. A backend with effective weight 0 is not "very unlikely", it is absent: it does not accumulate score and cannot win. And the current scores are long-lived state that survives from pick to pick; only the weights fed in change as time advances. A backend that rejoins (ADD or UP) starts a fresh life: its score goes back to 0.

Common mistakes

• Rounding the weight up or to nearest. 10 x 5 / 20 is 2.5, and the answer is 2.
• Letting a weight-0 backend into the rotation, even with the smallest chance. It must not win a single pick before its weight is above 0.
• Ramping the initial pool. POOL members have no ramp; only ADD and UP carry one.
• Resetting all scores on every pick, which throws away the interleaving smooth WRR exists to provide.
• Forgetting the cap: elapsed beyond the ramp must not exceed the target.

Your exercise

The starter holds the backends and parses the commands. You write the effective-weight function and the pick. POOL <name>=<w> ... replaces the pool at full weight, NOW <seconds> sets the clock, ADD <name>=<w> <ramp> joins at the current time with a ramp (replacing a backend of the same name in place, or appending), DOWN <name> takes a backend out, UP <name> <ramp> brings it back ramping from now, WEIGHTS prints <name>=<effective> sorted by name, and PICK prints the winner or NONE if nothing participates.