Putting It All Together

You've built every layer:

The full request pipeline:

Client request
  -> TLS termination (decrypt)
  -> Routing (hostname / path -> pool)
  -> Rate limiting (per client / global)
  -> Backend selection
        ↳ Health-filtered pool
        ↳ Algorithm (RR / least-conn / sticky / hash)
        ↳ Circuit breaker check
  -> Forward to backend (with retries on failure)
  -> Record response (passive health, metrics, access log)
  -> Return to client

Production LBs add:

• HTTP/2 + HTTP/3 support (multiplexing, server push)
• WebSocket upgrade routing (long-lived connections)
• gRPC load balancing (per-RPC, not per-connection)
• DNS-based routing (geo, latency-based)
• Anycast IPs (BGP routing same IP to nearest LB)
• Auto-scaling (add/remove backends based on load)
• Blue/green deployments (route X% of traffic to new version)
• Canary releases (10% to new version, watch metrics, ramp up)
• Request retries with jitter (avoid thundering herd)
• Hedged requests (send to two backends, take the first response)

You now understand what NGINX, HAProxy, AWS ALB/NLB, GCP Load Balancer, Envoy, and every other LB actually does. The decisions vary; the core patterns are universal.