Load Balancing on Lookup

When a client looks up billing, the registry has many instances. Which one do they use?

Random: pick any. Simple; statistically even.

Round-robin: rotate through instances. Even distribution but coordinated state needed.

Weighted: each instance has capacity weight. Pick proportionally.

Latency-based: track round-trip time per instance; prefer fast ones. Used by Envoy.

Subset shuffling: each client randomly picks K instances; load balances among those. Reduces fan-out.

Consistent hashing: same client → same instance. Useful for cache locality.

Server-side LB (kube-proxy, Envoy): the LB decides. Client just connects to LB address.

Client-side LB (Eureka, Consul DNS): client gets list, picks. Each client maintains its own state.

For most modern systems: client-side or service-mesh. Reduces hops; eliminates LB as bottleneck.

Watch out:

• Hot spots: random + few instances → some get hammered
• Subsetting: with 1000 clients × 100 instances, EACH instance gets connections from ~100 clients. Subsetting limits this.
• Stickiness vs spreading: tie-break with care

Production discovery (Consul, etcd, Eureka) all support multiple modes. Pick based on workload.