Tiered Limits

Production APIs never run just one limiter; every request passes through a stack:

1. Per-endpoint global: /login = 1000/sec across everyone  (capacity guard)
2. Per-IP:              5/sec                              (abuse guard, works pre-auth)
3. Per-user:            60/min                             (fairness guard)
4. Per-plan:            free = 100/day, pro = 10,000/day   (business rule)

A request is served only if every tier allows it — the tiers are ANDed:

Ordering is an engineering decision

Two competing principles:

• Cheapest first. In-process checks (a per-IP dict lookup) before networked ones (a per-plan counter in Redis). A flood from one IP then gets rejected without ever touching Redis — the limiter itself must survive the attack it's blocking.
• Strictest / most specific first. For the reason the next section makes painful.

The trap: early tiers pay for late rejections

allow() consumes quota as a side effect. In the ANDed chain, a request that passes tiers 1–2 and dies at tier 3 has still incremented tiers 1 and 2. Concretely: per-IP = 5/s, per-user = 2/s, and one user fires 5 requests in a second. Requests 1–2 pass both tiers and get served. Requests 3–5 increment the per-IP counter to 5/5, then die at per-user. Net effect: two requests served, but the IP's budget is fully spent — and a second user behind the same NAT is now denied by someone else's rejected traffic.

Your options, in rising order of effort: order the strictest tier first (per-user before per-IP here, so rejections happen before the shared counter is touched); do a read-only precheck of every tier before committing any increments (two passes); or accept the skew — which is what most production systems do, because refunding counters atomically across tiers costs more than the unfairness it repairs. Whichever you pick, pick it on purpose.

Tell the client which wall they hit

A bare 429 from a four-tier stack is undebuggable — the client can't distinguish "slow down for a minute" (per-user) from "you're done for the day" (per-plan). Return the binding tier and its retry hint:

HTTP/1.1 429 Too Many Requests
Retry-After: 86400
X-RateLimit-Triggered: per-plan

X-RateLimit-Triggered is a custom header, not a standard — what matters is the practice of naming the limit. GitHub does it: REST responses carry x-ratelimit-resource naming which budget ("core", "search", …) you exhausted.

Cost-based budgets

Not every request costs you the same. GitHub's GraphQL API charges each query points based on how many nodes it touches, against an hourly point budget — one deeply nested query can cost hundreds of trivial ones. Mechanically this is the weighted token bucket from the Token Bucket lesson: allow(client, now, cost) deducts cost tokens. Stripe's engineering blog describes the same layered philosophy in production: a request rate limiter plus a concurrency limiter plus load shedders, each guarding a different resource.

Your exercise

Build a two-tier stack — per-IP FixedWindow(limit=5, window=1) then per-user FixedWindow(limit=2, window=1), checked in that order — and fire 5 requests from (ip1, alice) at t=0. Verify: verdicts ALLOW ALLOW DENY DENY DENY, per-IP count = 5, alice's count = 2 (each tier refuses before incrementing itself, but per-IP already counted the requests per-user later refused). Now send 3 requests from (ip1, bob) in the same second: bob is brand new, yet gets DENY DENY DENY — alice's rejected traffic exhausted the shared IP budget. Reorder per-user first and rerun the whole scenario: bob now gets ALLOW ALLOW DENY. Same limiters, same traffic, different fairness — that reordering is the lesson.