Putting It All Together

You now hold every piece. First, the algorithm cheat sheet:

Where the limiter lives — defense in depth

• Edge (CDN / WAF): volumetric attacks, bots, and DDoS die here, before they touch your servers or your bill. Coarse rules — per-IP, per-ASN.
• Gateway (nginx, Envoy, Kong, AWS API Gateway): per-route, per-key policy. Note the algorithms you now recognize: nginx limit_req is a leaky bucket; Envoy's local limiter is a token bucket; the gateway is usually where 429s and Retry-After get attached.
• Application: limits only your code can express — "3 password resets per account per day," "one export job per workspace at a time." The outer layers can't see business objects.

Each layer catches what the outer one couldn't understand. The edge doesn't know your users; your app should never burn CPU on a flood the edge could have absorbed.

The production checklist

• Shadow mode first. Deploy the limiter logging would-be 429s without enforcing for a week. This is the single cheapest way to discover that your biggest customer's nightly sync would have been blocked on day one.
• Soft + hard thresholds. Warn (log, header, email) at 80% of budget; block at 100%.
• Adaptive shedding. When p99 latency or queue depth rises, tighten limits automatically — protecting the system beats honoring nominal quotas during an incident.
• Allowlists and denylists. Health checks and internal partners bypass; known abusers get zero.
• Humans vs machines. On human-facing endpoints a CAPTCHA challenge often beats a bare 429.
• Observability per tier. Which tier fires most, top denied clients, 429 rate over time. A sudden 429 spike is either an attack or your own misconfigured limit — you want to know which in minutes.

The client's half of the contract

A limiter only works with cooperating clients: honor Retry-After, back off exponentially with jitter (a thousand clients retrying on synchronized schedules re-create the flood), and watch X-RateLimit-Remaining to slow down before the wall. If you ship an SDK, build this in — Stripe's and GitHub's official clients both do.

Study list

Three excellent, public, real-world references: Stripe's engineering post on scaling with rate limiters (token bucket + layered shedders), GitHub's documented REST hourly quota and GraphQL point system, and Cloudflare's post on their sliding window counter. Everything in them should now read as familiar.

Your exercise

Design the full stack for an API of your own on paper: edge per-IP fixed window 1,000/min; gateway per-key token bucket capacity=20, refill_rate=5; app-level "3 password resets per account per day." Then verify the layers don't fight, with arithmetic: a legitimate key bursting 20 then sustaining 5/s sends at most 20 + 5 × 60 = 320 requests in its first minute — comfortably under the 1,000/min edge cap, so the edge never falsely fires on traffic the gateway would allow. Sizing outer limits so they only catch what inner limits could never permit is the last skill of this course: if your edge cap were 200/min, your own gateway policy would be unreachable — an inconsistency arithmetic catches before production does.