API Rate Limiting: A Complete Guide to Limits, Bursts and Backoff

An API without a rate limit works beautifully right up to the moment one client writes a loop with no delay in it. Then a single caller consumes the capacity everyone shares, latency climbs for every other user, and the failure looks like an outage rather than one misbehaving script. A rate limit is the mechanism that turns that scenario into a small, contained, well-labelled error for one client.

Getting limits right is mostly a matter of being explicit: deciding what you are counting, over what period, per whom, and what you tell a client when it goes over. This guide covers the algorithms, how to choose numbers you can defend, the headers that let a well-written client stay inside your limits, and how to back off when you are the client.

Why Rate Limiting Matters

Rate limiting is usually introduced after an incident, but it solves four separate problems at once:

• Capacity protection. Your service has a maximum sustainable throughput. A limit keeps demand below it by design rather than by luck.
• Fairness. Without a per-client limit, the noisiest caller gets the most resources. With one, every client gets a predictable share.
• Cost control. If each request triggers a database query, a third-party call or a model invocation, an unlimited API is an unlimited bill.
• Abuse resistance. Credential stuffing, scraping and enumeration all depend on volume. A limit makes them slow enough to be impractical and visible enough to alert on.

There is a fifth, quieter benefit: a documented limit is a contract. Clients can be built to respect it, which is far better than discovering your ceiling by hitting it.

The Algorithms Worth Knowing

Every rate limiter answers the same question — is this request allowed right now? — but they differ in how they treat bursts, and that difference is what you actually feel as a client.

Token Bucket

Picture a bucket that holds a fixed number of tokens and refills at a steady rate. Each request removes one token. If the bucket is empty, the request is rejected. Two numbers describe it completely: the refill rate, which is your sustained throughput, and the bucket size, which is the largest burst you will tolerate.

A bucket of 100 tokens refilling at 10 per second lets an idle client fire 100 requests immediately, then settles it to 10 per second. That is usually what you want: real clients are bursty, so a short spike is normal traffic rather than abuse, while the long-run average stays bounded. Token bucket is the default for good reason — it is cheap to implement, needs only a token count and a timestamp per client, and its behaviour is easy to explain in documentation.

Leaky Bucket

The leaky bucket is the mirror image. Requests go into a queue and leave at a constant rate, like water draining through a hole. If the queue is full, new requests are rejected. The output is perfectly smooth, which is exactly what you want in front of a backend that cannot absorb spikes at all, such as a legacy system or a third-party API you are reselling.

The trade-off is latency. A request that arrives into a full-ish queue waits instead of failing fast, and a client cannot tell a slow server from a queued request. Use it to shape traffic into something fragile; prefer token bucket when you are protecting your own elastic service.

Fixed and Sliding Windows

A fixed window counts requests per calendar period: 1,000 per hour, reset on the hour. It is trivial to implement with a counter and a key, and it has one well-known flaw: the boundary burst. A client that sends 1,000 requests at 10:59:59 and another 1,000 at 11:00:01 has stayed inside the policy while delivering 2,000 requests in two seconds — twice your intended peak.

A sliding window fixes this by counting over the trailing period rather than the current calendar block. At 11:00:01 it still sees the requests from 10:59:59 and rejects the excess. The accurate version stores a timestamp per request, which costs memory; a common compromise weights the previous window's count by how much of it still overlaps, which is close enough for almost all APIs at a fraction of the state.

Choosing Your Limits

Pick numbers from measurements, not from a round figure that looks generous. A workable sequence:

1. Measure your ceiling. Load-test until latency degrades. That throughput, minus headroom, is your real capacity.
2. Divide by expected concurrent clients. This is the starting point for a per-client limit, not the final answer.
3. Look at your current traffic. Find the 95th and 99th percentile request rate per client. A limit below the 99th percentile will page you about legitimate users.
4. Add a burst allowance. Roughly 10 to 20 times the per-second rate is a common bucket size; it absorbs normal page loads without permitting sustained abuse.
5. Tier it. Anonymous, authenticated and paid clients should not share a limit. Charging more for a higher ceiling is also the cleanest way to price an API.
6. Observe before enforcing. Log what would have been rejected for a week before you start rejecting it.

What you key on matters as much as the number. Per-IP is the only option for anonymous traffic but punishes shared networks and NAT; per-API-key is fair and the right default for authenticated calls; per-account stops a client from buying extra capacity by minting keys. Many APIs apply all three at once, as a global limit, a per-key limit and a per-endpoint limit on the expensive routes.

Once you have a number, the arithmetic of turning it into a request interval is easy to get subtly wrong. The Rate Limit Calculator converts a limit and window into the minimum delay between requests, the safe sustained rate and a Retry-After value you can publish.

The Headers Clients Need

A limit a client cannot see is a limit a client will hit. Send your state on every response, not only on rejections:

• RateLimit-Limit — the ceiling for the current window.
• RateLimit-Remaining — how many requests are left in it.
• RateLimit-Reset — when the allowance refreshes, in seconds from now.
• Retry-After — on a 429 or 503, how long to wait. Either a number of seconds or an HTTP date.

The X-RateLimit-* spellings are still widespread and perfectly workable; the unprefixed names above are the standardised form, and sending both during a transition costs nothing. The important part is that the values are present and accurate, because they let a good client pace itself and never see a 429 at all.

When you do reject, return 429 Too Many Requests, never 403 and never a generic 500. A 429 is the one code that tells a client the request was fine and will succeed later, which is the only signal that produces correct retry behaviour. Our HTTP Status Codes reference covers the related codes and what each one obliges a client to do.

Handling a 429 as a Client

From the other side of the connection, the rules are short and almost always under-implemented:

1. Respect Retry-After. If the server tells you how long to wait, wait that long. Nothing else you do is better informed.
2. Otherwise back off exponentially. Wait 1s, 2s, 4s, 8s, up to a cap. Retrying immediately turns one overload into a self-inflicted stampede.
3. Add jitter. Randomise each delay by up to a few hundred milliseconds. Without it, every client that failed together retries together, and your backoff schedule reproduces the spike it was meant to spread.
4. Cap the attempts. Three to five retries, then surface the failure. Infinite retries hide a real outage.
5. Pace yourself from the headers. Read RateLimit-Remaining and slow down before you are rejected. This is the difference between a client that works and one that works most of the time.
6. Never retry a 4xx other than 429. A 400 or 422 will fail identically every time, because the request itself is the problem.

Common Mistakes to Avoid

• Returning 403 instead of 429. Clients read 403 as permanent and stop, or retry blindly; neither is the behaviour you want.
• Omitting Retry-After. You have asked the client to guess, and the common guess is "immediately".
• Rate limiting on IP alone. One office behind a single NAT address becomes one client, and a distributed scraper becomes a thousand.
• Counting every endpoint the same. A cached read and a report that runs for eight seconds should not cost one token each; weight expensive routes or give them their own limit.
• A fixed window with a tight limit. The boundary burst means your true peak is double the published number.
• No burst allowance at all. A perfectly smooth limit rejects normal traffic, because real clients arrive in clumps.
• Undocumented limits. If the number is not in your documentation and your headers, every integrator discovers it in production.
• Retrying without jitter. Synchronised retries are how a brief spike becomes a sustained one.

Work Out Your Own Limits

A good rate limit is boring: clients pace themselves from your headers, bursts are absorbed, abuse is throttled, and nobody notices. Getting there is mostly arithmetic and honesty about your real capacity. Start from a limit and window you can actually sustain, convert it into the interval and Retry-After values your clients need, publish both, and log near-misses so you can tell a limit that is too tight from a client that is misbehaving.

Related tools and guides:

• Rate Limit Calculator - turn a limit and window into intervals, burst sizes and Retry-After.
• HTTP Status Codes - what 429, 503 and the rest oblige a client to do.
• HTTP Status Codes: A Complete Guide - every response class explained.
• DNS Lookup - confirm which host your API traffic actually reaches.

Frequently Asked Questions