Scrapeless Wiki

How Does Rate Limiting Work?

Informational P2 how does rate limiting work
Overlaps a live page. A published page already targets this keyword: https://www.scrapeless.com/en/blog/rate-limiting. Publishing both risks cannibalisation — decide before this goes live.

Learn how rate limiters are implemented: counters, token buckets, sliding windows, where the state lives, and how distributed limits stay consistent.

A rate limiter answers one question on every request: has this client already used its allowance? The interesting part is how that question is answered cheaply, and how it stays correct across many servers.

1. The Basic Shape

  1. Derive a key from the request — API token, IP address, account, or a combination with the endpoint.
  2. Look up the counter state for that key.
  3. Decide whether the request fits within the policy.
  4. Update the state.
  5. Allow and add headers, or reject with 429.

Steps 2 and 4 must be atomic. If two concurrent requests read the same counter before either writes, both are admitted and the limit is exceeded — a race that appears exactly when load is high enough to matter.

2. Fixed Window

The simplest implementation: one counter per key per time window.

key:  ratelimit:user123:202608201430
value: 47
TTL:   60 seconds

Increment on each request; reject above the threshold; let the key expire.

Cheap and easy, with a known flaw: boundary bursting. With a limit of 100 per minute, a client can send 100 at 11:59:59 and 100 more at 12:00:00 — 200 requests in one second, entirely within policy. Whether that matters depends on whether you are protecting an average or a peak.

3. Sliding Window

Two refinements fix the boundary.

Sliding log stores a timestamp per request, drops entries older than the window, and counts what remains. Exact, and memory grows with request volume.

Sliding window counter interpolates between the current and previous fixed windows:

count = current + previous × (fraction of previous window still in range)

Approximate, constant memory, and it eliminates boundary bursting in practice. This is what most production limiters use.

4. Token Bucket

The most common algorithm for public APIs, and the most intuitive.

A bucket holds tokens up to a capacity. Tokens refill at a fixed rate. Each request spends one; an empty bucket means rejection.

capacity:    100 tokens
refill rate: 10 tokens/second

An idle client accumulates tokens up to the capacity and can then burst. A sustained client is held to the refill rate.

The elegance is that both limits are expressed in one structure: capacity is the burst allowance, refill rate is the long-run average. That matches how real clients behave — mostly quiet, occasionally busy.

Implementation is two numbers, not a list: the token count and the last refill timestamp. Tokens are computed lazily on read.

5. Leaky Bucket

Requests enter a queue and drain at a fixed rate. Output is perfectly smooth, and the queue absorbs bursts rather than rejecting them.

Where token bucket permits bursts, leaky bucket flattens them. Useful when the protected resource cannot tolerate spikes at all, at the cost of added latency for queued requests.

6. Distributed Limiting

One server with an in-memory counter is easy. Ten servers behind a load balancer is the real problem — ten local counters of ten each permit a hundred.

Shared store. Redis holds the counters; every server reads and writes it. Correct, and it adds a network round trip to every request and makes Redis a dependency of serving traffic.

-- atomic increment-and-check in one round trip
local n = redis.call('INCR', KEYS[1])
if n == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]) end
return n

The Lua script matters: INCR and EXPIRE as separate calls can leave a key with no expiry if the process dies between them, permanently exhausting that client's quota.

Local approximation. Each server enforces limit ÷ N. No coordination, and unfair when load distributes unevenly.

Sync periodically. Local counters reconciled every few seconds. Fast, with a window of over-admission.

At the edge. A CDN or gateway limits before traffic reaches your servers — which is why Cloudflare's rate limiting produces a 1015 that your origin never sees.

7. Communicating the Limit

HTTP/1.1 429 Too Many Requests
Retry-After: 30
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 1735689600

Emitting these on successful responses too is what allows well-behaved clients to slow down before they are rejected. A limiter that only speaks when refusing forces every client to discover the boundary by crossing it.

8. Summary

A rate limiter keys each request to an identity, checks and updates a counter atomically, and allows or rejects. Fixed windows are cheapest and permit boundary bursts; sliding windows fix that exactly or approximately; token bucket expresses burst allowance and sustained rate in one structure and dominates public APIs; leaky bucket smooths output completely.

Across multiple servers the state must be shared — usually Redis with atomic scripts — or approximated locally, trading accuracy for latency. Edge limiting moves the decision in front of the origin entirely.

The detail that most improves client behaviour costs almost nothing: send the limit headers on successful responses, so clients can throttle before they hit the wall rather than after.