Metrics & Observability

A load balancer is the one component that sees every request — which makes it the best vantage point in your whole system. Instrument it well and you can answer "is the site up?", "which backend is slow?", and "when do we need more capacity?" without touching a single application.

The industry-standard model (Prometheus) has three metric types, and choosing the right one is most of the skill:

Counters only go up: lb_requests_total, lb_bytes_in_total, lb_health_check_failures_total. You never read a counter directly — you ask for its rate (rate(lb_requests_total[5m])). Because rates are computed from differences, a counter that resets to zero when the LB restarts does no real harm — which is exactly why "total since start" beats storing a gauge of requests-per-second yourself.

Gauges go up and down and are read directly: lb_active_connections, lb_backend_health (1 or 0), lb_pool_size.

Histograms record distributions in buckets: lb_request_duration_seconds. Buckets let the server compute p50/p99 across all LB instances — the reason histograms beat summaries, which pre-compute percentiles per-instance and can't be aggregated. Averages hide everything interesting: an average of 50 ms is compatible with a p99 of 4 seconds.

If you remember one framework: RED — Rate, Errors, Duration — is the minimal dashboard for any request-driven service, and all three come straight from the metrics above.

Labels are the power tool with a safety warning. lb_requests_total{backend="s1",status="500"} lets you slice by backend and status — but every distinct label combination is a separate time series. Label by backend (tens) and status (a handful), never by user ID or raw URL path (unbounded). Cardinality explosions are the classic way to kill a Prometheus server.

The exposition format is plain text, one metric per line:

lb_requests_total{backend="s1",status="200"} 4823
lb_requests_total{backend="s1",status="500"} 12

Completing the observability picture: access logs (one line per request, for debugging individual failures), traces (propagate traceparent, add the LB as a span), and the LB's own endpoints — /healthz (am I alive?), /readyz (should I receive traffic? — flipped off during startup and drain), /metrics (the scrape target).

Your exercise

Implement record_request(), record_bytes(), and emit() in the starter's MetricsAggregator. METRICS must print five blocks in this exact order, each internally sorted:

1. lb_requests_total{backend="<b>",status="<s>"} <count> — one line per (backend, status) pair that actually occurred, sorted by backend then status. Never-seen pairs are omitted.
2. lb_bytes_in_total{backend="<b>"} <n> — one line per pool backend, sorted, even when 0.
3. lb_bytes_out_total{...} — same rule.
4. lb_latency_avg_ms{backend="<b>"} <avg> — per pool backend, sorted; average request duration truncated to an integer (0 if no requests).
5. lb_health{backend="<b>"} <1|0> — per pool backend, sorted; default 1.

Grader traps:

• The always-emit rule. A pool backend with zero traffic still gets its bytes/latency/health lines: POOL only followed immediately by METRICS prints exactly 4 lines (lb_bytes_in_total ... 0, lb_bytes_out_total ... 0, lb_latency_avg_ms ... 0, lb_health ... 1) — and no lb_requests_total line, because no (backend,status) pair ever occurred.
• Truncate, don't round. Durations 25 and 30 average to 27 (27.5 truncated), not 28. Integer division (sum // count) is your friend.
• Exact label syntax. Values are quoted (backend="a"), no space before the {, one space before the number. Copy the format strings from the examples; the diff is byte-exact.