What is prompt caching (in plain English)?
Plain-English definition · Updated 2026-10-29. Multipliers below are the providers' own published list rates as of this date and are stated relative to each provider's base input price, so they survive a price cut; the dollar examples are our arithmetic on those rates, not vendor claims.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
Prompt caching is how every major API sells you the same prompt prefix twice — once at full price, once nearly free. The provider stores the processed state of a repeated prefix (system prompt, tool definitions, a pasted document) and charges a much lower rate when the next request re-reads it. In 2026 the four biggest APIs agree on the shape and disagree on the price: Anthropic charges 1.25× input to write a 5-minute cache (2× for 1 hour) and 0.1× to read it; OpenAI also charges 1.25× to write (30-minute life, 2.5× for 12 hours) and 0.1× to read on its current models; Google bills cached tokens at about 10% of input and adds a storage meter of $0.50 per million cached tokens per hour; DeepSeek charges $0.003 against $0.15 — a 98% discount with no write or storage fee at all. The buyer's rule: cache only prefixes you will reuse at least twice, keep the stable part first and the variable part last, and never assume a hit — an exactly identical prefix is the price of the discount.
What it actually means
Every request to a hosted model begins the same way: the provider tokenizes your input and runs a forward pass over the whole prompt before it can emit the first output token. That pass is the expensive, latency-heavy part, and in agentic workloads it repeats almost verbatim — the same system instructions, the same dozen tool schemas, the same 40-page policy document, every single turn. Prompt caching stores the result of that pass, keyed to the exact prefix, so the next request can skip it. It is the server-side cousin of the KV cache: same underlying state, but persisted by the provider across requests instead of inside one conversation.
The mechanism is prefix matching, not semantic matching. Anthropic's documentation is explicit that a hit requires 100 percent identical prompt segments up to the block you marked; OpenAI and Google describe the same requirement in their own words, and all three isolate caches per organization so nobody benefits from your prefix by accident. That single constraint generates most of the practical advice in this entry: anything that changes early — a timestamp, a per-user greeting, a reordered tool list, even non-deterministic JSON key order in some languages — breaks the prefix and you pay full price for the whole prompt again. Where you put the variable part of your request is not style, it is the billing switch.
Billing splits into three meters, and the split is where buyers get surprised: a write (sometimes called cache creation), a read (a hit), and ordinary input for the uncached remainder. Anthropic and OpenAI both price the write above base input and the read far below it; Google prices the read near 10% but charges rent on the stored tokens by the hour; DeepSeek charges neither a write premium nor storage, only a much cheaper hit rate. Read that as three different bets on how long your prefix will live.
Why it matters when you're picking a tool
One number decides whether caching pays: how many reads per write. Take a mid-tier model at $3 per million input tokens and a 100,000-token system prompt you reuse across a 20-turn agent session. No caching: 20 × 100K × $3/M = $6.00. With a 5-minute-style cache (1.25× write, 0.1× read): one write at $0.375 plus 19 reads at $0.03 each = $0.945 — about 6.3× cheaper, from a code change that only moves the static block to the front. Do the arithmetic on your own session length before you compare vendors: a workload that reuses a prefix three times barely beats the write premium, and a chat product with a per-user system prompt never gets a hit at all.
The write premium is a one-off; the storage meter is not. Anthropic's 5-minute write costs 1.25× and its own documentation states the cache pays for itself after one read (two reads for the 1-hour, 2× write). OpenAI's current models follow the same shape — 1.25× to write a 30-minute cache, 0.1× to read — with 2.5× for the 12-hour variant. Google is structurally different: on Gemini's published rates a cached token reads at about a tenth of input and accrues $0.50 per million tokens per hour of storage. Our arithmetic on those two numbers: you save $0.675 per million tokens per read, so a million cached tokens must be read more than 0.74 times an hour — roughly once every 80 minutes — just to pay its own rent. Great for a hot document behind a busy endpoint; quietly expensive for a cache you keep warm overnight and query twice.
Minimum lengths quietly exclude the small stuff. Anthropic publishes a per-model floor that ranges from 512 tokens on its newest models to 4,096 on several older ones; OpenAI's current guidance is 1,024 tokens for its latest generation, and older models varied with request settings. Below the floor the request is processed normally and simply is not cached — no error, no warning, just a bill with no discount. That is why caching changes the economics of long-document and agent workloads and does nothing for a 200-token classification prompt, no matter which vendor you pick.
The 2026 reality check
The discount got much bigger than the launch press releases. OpenAI's 2024 announcement framed prompt caching as a 50% discount with a 1,024-token minimum and 128-token increments. Its current documentation describes cached reads at 0.1× input (0.05× on two of its newest models) and a write premium of 1.25× or 2.5× depending on cache life — the same shape Anthropic has used since 2024. Anyone still budgeting 50% off is overstating their bill by roughly 5× on the cached portion. Check the live docs, not the blog post that ranks first.
"Automatic" is not the same as "free". OpenAI applies caching without any code change and DeepSeek has done the same on disk since 2024, which reads as zero engineering work — and it is, for structure. But automatic caching still only helps if your prefix is stable, and both vendors isolate caches per organization so a shared template across customers buys you nothing. Anthropic's model is the opposite trade: you mark breakpoints explicitly (up to 4 of them), you pay the write premium, and you get the deepest read discount plus free refresh every time the cache is used. Explicit control is worth the code when your prefix is long and its reuse is predictable; automatic is worth the simplicity when it is not.
The cheapest per-token vendors are also the cheapest to cache into. DeepSeek's published rate card puts cache hits at $0.003 per million tokens against $0.15 for a miss on its flash model — a 98% gap — with no write premium and no hourly storage, and it runs the same automatic prefix match with nothing to configure. Compare that to Google's rent meter before assuming the two are interchangeable: a workload with a large prefix read a few times an hour will look cheap on DeepSeek and expensive on Gemini, and the entire difference lives in the storage line, not the token line. This is the same trap as inference cost in general — the headline rate is rarely the bill.
Caching changes latency, and latency changes product design. Both Anthropic and OpenAI describe caching as cutting the time spent processing input before the first output token, and OpenAI documents a pre-warm call to populate a cache ahead of traffic. That is why agent products can afford to re-send a 100K-token context on every step: without the cache the round trip would be dominated by reprocessing, and with it the model effectively starts from the last token. If a competitor's assistant feels materially snappier on long documents, this — not a faster model — is often the explanation.
Quick checklist
- Count reads per write before you build. One reuse clears a 1.25× write; the 1-hour and 12-hour variants need two or more. If your prefix changes per user or per request, caching cannot help and you should not pay for the plumbing.
- Put the stable block first and the variable block last. Prefix matching is exact: a timestamp, a user name or a reordered tool list at the top of the prompt converts every request into a full-price miss.
- Check the minimum length for your exact model. Floors run from 512 to 4,096 tokens depending on model generation. Under the floor you get no cache and no error — only an undiscounted invoice.
- Price the storage meter, not just the read rate. Rent-based caching needs roughly one read per 80 minutes for a million cached tokens at Google's published $0.50/hour. A cache you keep warm and rarely query is a subscription, not a saving.
- Do not assume a 50% discount. Current list rates put cached reads near a tenth of input (sometimes a twentieth). Budget from the live docs, and re-check after every model launch — multipliers survive price cuts, absolute rates do not.
- Keep cached and uncached traffic separately metered. If your cost model collapses everything into "input tokens", your savings estimate is fiction — track writes, reads and fresh input as three lines.
Where you'll hit it
Three purchases on this site are really decisions about prompt caching: ChatGPT vs Claude (both ship aggressive cached-input pricing and both document a different control model — automatic versus explicit breakpoints), the best AI chatbots ranking (where "handles long documents" is partly a claim about cache economics), and Ollama vs LM Studio — locally you already get prefix reuse for free, which is the honest reason a home setup can feel competitive on repeated long context. Related terms: KV cache is the same state priced per token instead of per request; AI token is the unit all three meters are counted in; context window is the budget the cached prefix occupies. Full list on the AI tools glossary.
The bottom line
Prompt caching is the rare AI feature that is purely arithmetic: store a repeated prefix, pay 1.25× once, read it back at a tenth, and a 20-turn agent session on a 100K-token context drops from about $6.00 to about $0.95. The 2026 pricing has converged on that shape — Anthropic and OpenAI both at 1.25× write and 0.1× read, DeepSeek at 2% with no premium at all — while Google keeps a rent meter of $0.50 per million cached tokens per hour that no other major provider charges. So the questions worth asking any API vendor are boring and specific: what is the minimum cacheable length for this model, is the write premium one-off or renewed, is there a storage charge, and does the discount survive my per-user prompts? Answer those four and you will know whether caching saves you 80% or sells you a subscription you never use.