Least-Connections

Round-robin assumes every request costs the same. Real traffic is nothing like that: a health ping returns in 2 ms while a report export holds its connection for 2 seconds. Under round-robin the backend that keeps drawing the expensive requests falls behind while its neighbours idle. The request count is even but the load is not. In this lesson your balancer starts measuring load instead of guessing it.

The mechanism

Least-connections uses the balancer's own bookkeeping as a live load signal. It keeps one counter per backend, the number of requests it has sent there that have not finished yet (the in-flight count), and dispatches each new request to the backend with the smallest count.

pseudocode
  pick():
      best = the backend with the smallest in_flight (first one listed wins a tie)
      in_flight[best] += 1            # count it the moment it is dispatched
      return best

  done(backend):
      if in_flight[backend] > 0: in_flight[backend] -= 1

A backend that is slow, for any reason (GC pause, cold cache, noisy neighbour), automatically accumulates open requests and stops receiving new ones until it drains. That feedback loop is the whole point, and it costs nothing: the counts are numbers the balancer already has.

A worked trace on three backends, with the in-flight counts after each event:

event        a  b  c    chosen
pick         1  0  0    a
pick         1  1  0    b
pick         1  1  1    c      <- all tied again
done b       1  0  1
pick         1  1  1    b      <- b is the only one with 0
pick         2  1  1    a      <- three-way tie, first listed

Where it is used

• Long-lived connections. A WebSocket or gRPC stream can live for hours. Round-robin would stack immortal streams onto whichever backend was slow at connect time; least-connections corrects itself.
• HAProxy calls it balance leastconn, nginx calls it least_conn, and AWS ALB's version is least outstanding requests: same idea, counting in-flight HTTP requests.
• The counts are the balancer's local view. With several balancers in front of one pool, each sees only its own traffic, which is the problem the next lesson turns into a feature.

Ties

At startup every backend has 0 in flight, so everything is a tie. Production balancers rotate among the tied backends. Always taking the first would send a whole initial burst to backend 1. This exercise uses the simpler rule, the first tied backend in pool order, so the output is deterministic. Know that the production answer differs; a quiz will ask.

Common mistakes

• Counting too late. Increment the winner before the next decision. Ten simultaneous decisions that all see 0 will pile onto one backend. Here that shows up as three picks on POOL a b c printing a a a instead of a b c.
• Going negative. A DONE for a backend with nothing in flight must stay at 0. A count of -1 makes that backend win every future pick.
• Mixing up the two orders. Ties are broken by pool order; STATUS is printed sorted by name. A pool listed as zeta alpha mid is where the two come apart.
• Crashing on strangers. DONE for a name that is not in the pool is ignored, and PICK on an empty pool answers EMPTY.

Your exercise

The starter reads the commands and prints the results; you write the bookkeeping behind pick and done. POOL <b1> <b2> ... replaces the pool with all counts at 0 (prints OK). PICK chooses, counts and prints the backend. DONE <backend> finishes one request (prints OK). STATUS prints one <backend>:<in_flight> line per backend, sorted by name.