What are API rate limits (and why does your AI tool slow down)?
Plain-English definition · Updated 2026-11-02. Every rule below is quoted from OpenAI's and Anthropic's own rate-limit documentation; the comparisons between the two are ours.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
API rate limits are the ceilings a model provider puts on how much your application may send per unit of time — requests and tokens per minute, sometimes per day, sometimes images. OpenAI defines them as "restrictions that our API imposes on the number of times a user or client can access our services within a specified period of time" and measures five dimensions (RPM, RPD, TPM, TPD, IPM); Anthropic measures three (RPM, ITPM, OTPM) per model class. Both are enforced per organization, not per user — OpenAI states it outright: "Rate limits are defined at the organization level and at the project level, not user level." Exceed one and you get HTTP 429, which is where most teams misread the situation: some 429s carry a retry-after and clear in seconds, others carry none and will not clear until next month. The practical consequence is that rate limits behave less like a price and more like a capacity contract — the thing that decides whether your product can serve the traffic you already sold.
What it actually means
The mechanism is unglamorous: a provider meters your traffic on several axes at once and rejects requests when any single axis is exhausted. OpenAI's documentation puts it exactly that way — "Rate limits can be hit across any of the options depending on what occurs first" — across RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day) and IPM (images per minute), plus audio minutes for streaming audio models. Anthropic documents a narrower but sharper set — "measured in requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM) for each model class" — which means the ceiling is not one number but one number per model family, and one family's headroom tells you nothing about another's.
The second structural fact is scope. Both providers aggregate at the organization, not at the end user: OpenAI's docs state limits are "defined at the organization level and at the project level, not user level," and Anthropic enforces service-configured limits at the organization level while letting you set lower per-workspace caps (with the warning that "Organization-wide limits always apply, even if Workspace limits add up to more"). Read that as a design constraint: a multi-tenant product shares one ceiling across all its customers, so a single heavy user can throttle everyone else, and no amount of per-user throttling in your own code changes the provider's arithmetic.
Third, the ceilings are not static. OpenAI tiers organizations automatically — "Your organization's usage tier upgrades automatically as its total credit purchases reach each threshold" — with tiers named Free, Build, Launch and Grow in its current docs, each unlocking higher RPM and TPM as well as a higher monthly usage cap. Anthropic does the same with usage tiers assigned from "usage history and account standing," and notes that new organizations "may start in the Evaluation tier, with limits below the standard limits shown on this page." Translation for a buyer: the rate limit you read on the pricing page is the limit you have today, not the one a funded competitor has.
Why it matters when you're picking a tool
The 429 is not one error. OpenAI returns 429 with type rate_limit_error and code slow_down, and — this is the part worth memorising — "A slow_down error can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits. It reflects how quickly traffic increased, not whether you exhausted those limits." Your dashboard can show green across every published limit while production is erroring, because the axis that tripped is acceleration. Anthropic's 429 is a different animal: it names the limit that was exceeded and, "when the response has a retry-after header, wait that many seconds before retrying."
And one 429 has no fix at all. Hit Anthropic's monthly spend cap and you also get 429 — but the docs say the response "has no retry-after header," the error code is enforced_spend_limit_reached, and "retrying, including the SDK's automatic retries, fails until access resumes" at 00:00 UTC on the first of the next month. Same status code, opposite remedy: one wants backoff, the other wants a billing change. A retry loop tuned only on status code will hammer a wall for days. This is the same failure shape as the invisible line item in our reasoning tokens entry — the bill and the error are both telling you something the UI does not.
The max_tokens knob moves two providers in opposite directions. OpenAI's guide is explicit that a request's token cost for limit purposes is "calculated as the maximum of max_tokens and the estimated number of tokens based on the character count of your request" — so a generous max_tokens reserves headroom against your TPM ceiling whether or not you use it. Anthropic documents the mirror image: "OTPM rate limits are evaluated in real time as output tokens are produced, counting only the actual tokens generated. The max_tokens parameter does not factor into OTPM rate limit calculations." Identical-looking parameter, opposite effect on throughput: on one platform an inflated cap costs you concurrent capacity, on the other it costs you nothing. If you port a prompt stack between providers without re-checking this, your throughput changes for reasons no profiler will show you.
Caching is a rate-limit lever, not just a discount. Anthropic's docs state that "for most Claude models, only uncached input tokens count toward your ITPM rate limits," with cache reads excluded (and cache writes counted). That makes prompt caching do double duty: it cuts the price and raises your effective input ceiling, because the tokens you re-send are no longer the ones being metered. OpenAI's guidance points elsewhere but reaches a similar conclusion for batchable work: the Batch API lets you "submit and execute large collections of requests without impacting your synchronous request rate limits."
The 2026 reality check
Retries are not free, and that is now documented. OpenAI's guide says it plainly: "unsuccessful requests contribute to your per-minute limit, so continuously resending a request won't work." A naive retry storm therefore consumes the very capacity it is waiting for — the opposite of the intuition that retries buy you reliability. Both providers push you toward exponential backoff with jitter, and OpenAI's own instruction is to treat Retry-After as a floor: "wait at least that long and add a small random delay so multiple clients don't retry at the same time."
Provisioning advice has become an engineering spec. OpenAI now tells customers that "once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes" — a ramp curve, not a purchase order. That single sentence is the clearest admission of what rate limits really are: a stability mechanism for the provider's fleet, sized to protect aggregate load (the docs list exactly that reason, alongside abuse prevention and fair access). You are not being priced; you are being scheduled.
Which is why "unlimited" plans are just rate limits with the numbers hidden. Consumer chat subscriptions meter you in messages per window rather than tokens per minute, and the two are not convertible without knowing how many tokens a message costs — the same ambiguity our AI token and inference cost entries work through. For API buyers the practical hierarchy is: token price sets the bill, rate limit sets the ceiling, and the two are independent. A cheap model you cannot call fast enough is more expensive than an expensive one you can — which is precisely the trade-off inside our best AI coding assistants ranking, where agentic harnesses fire many calls per task and hit per-minute walls before they hit budget walls.
Quick checklist
- Read the ceilings that apply to you. Tier names match; the numbers behind them do not. Confirm your organization's current RPM/TPM in the console before sizing a launch, not the marketing page.
- Log the 429 body, not just the status.
rate_limit_errorwith aretry-afteris transient;enforced_spend_limit_reachedwithout one is a billing event. Route them to different code paths. - Budget
max_tokensagainst throughput. On OpenAI an inflated cap reserves per-request headroom; on Anthropic it does not count at all. Set it from your real output distribution and re-check after any migration. - Assume one shared pool. Because limits are enforced per organization, one noisy tenant can starve the rest. Enforce your own per-tenant budget below the provider's ceiling rather than discovering the ceiling in production.
- Use caching and batching as capacity, not only as discounts. Cache reads do not count toward Anthropic's ITPM; batch traffic does not consume OpenAI's synchronous limits. Both raise throughput at the same list price.
Where you'll hit it
Every comparison on this site quietly assumes you can call the model as often as you like. Our ChatGPT vs Claude and Perplexity vs ChatGPT pages are really comparisons of two different ceilings: the consumer plans meter messages, the APIs meter tokens per minute, and a tool built on either inherits that arithmetic. Related terms: tokens per second is the speed you get within the ceiling, reasoning tokens are why one "message" can consume several thousand tokens of headroom, and KV cache is the memory pressure that makes long contexts expensive to serve in the first place. Full list on the AI tools glossary.
The bottom line
Rate limits are the part of a provider's contract that has nothing to do with price and everything to do with whether your product works at 3pm on a Tuesday. They are enforced per organization, measured on several axes at once, and can fire while every published number still looks green — because one of them measures acceleration rather than volume. The two facts worth carrying into any tool decision: a 429 with a retry-after is a delay, a 429 without one is a billing problem, and max_tokens costs you throughput on one major platform and nothing on the other. Size your launch against your tier's ceiling, cache and batch to buy headroom at list price, and treat the anthropic-side spend cap as the hard stop it is. The ceiling you never measured is the one that takes you down.