What is agent memory (and what does it actually cost)?
Plain-English definition · Updated 2026-11-17. Primary sources: Anthropic's memory-tool documentation and its 2025-09-29 context-management announcement, the Mem0 paper (arXiv 2504.19413) and Mem0's own pricing page, plus our own token arithmetic at published list rates. Numbers were re-checked on 2026-11-17.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
Agent memory is the layer that lets a model keep facts, preferences and task state across sessions instead of re-sending the whole conversation inside the context window every turn. In 2026 it is three different products sold under one word: conversation-replay session stores, a file-based memory tool the model edits itself (Anthropic's memory_20250818: six commands against a /memories directory, client-side storage, shipped 2025-09-29 with context editing — Anthropic reports 39% better performance with both, 29% from context editing alone, and 84% fewer tokens on a 100-turn search evaluation), and extracted-fact stores like Mem0, whose paper claims 26% relative gain over OpenAI's memory, 91% lower p95 latency and 90%+ fewer tokens than full-context replay — carrying ~7,000 tokens of memory instead of a ~26,000-token transcript. The buyer's trap: those savings are measured against full replay, and against prompt caching the arithmetic inverts. Memory is a window-boundary product, not a token-saving one.
What does agent memory actually mean?
Agent memory is the persistence layer that lets an AI agent retain what it learned in an earlier session and act on it in a later one — user preferences, project state, decisions already made — without paying to re-read the entire prior conversation on every request. The context window is working memory and it is erased at the end of a call; memory is the thing on the other side of that erasure. Everything else in the category is an implementation choice about what gets written down and who decides.
Those implementation choices split into three families, and they are not interchangeable products.
- Replay (session stores). The framework persists the message list and re-injects it — sometimes verbatim, sometimes with a rolling summary. This is the default in most agent SDKs because it needs no new infrastructure: the "memory" is the conversation, and the bill is the conversation's length. It works exactly until the transcript outgrows the window, which for an agent doing multi-hour work is measured in days, not months.
- Tool-based files. The model is given a small filesystem and decides what to write. Anthropic's memory tool is the cleanest published example: a
/memoriesdirectory, six commands (view,create,str_replace,insert,delete,rename), executed client-side — the model emits a tool call and your application performs the file operation against whatever store you choose. Anthropic frames it as "just-in-time" retrieval: keep the active context focused, fetch detail on demand. The design consequence is that the model owns relevance and your infrastructure owns durability; there is no ranking function to tune, and no bill except the tokens the model spends reading and writing. - Extracted facts. A separate pipeline reads each exchange, extracts candidate facts, and consolidates them against what is already stored — the Mem0 paper describes the update step as an LLM choosing among ADD, UPDATE, DELETE or NOOP for each candidate. Retrieval then returns a small ranked set per query instead of a transcript. This is the family that produces the benchmark numbers everyone quotes, and the one that carries a per-turn write bill nobody puts on the pricing page.
The distinction that decides purchases is who chooses what to forget. In replay, nobody does — the window truncates. In the tool model, the model decides, in-band, using the same judgment it uses for everything else. In the extracted-fact model, an extraction prompt decides, out-of-band, before you ever see the result. Those are three different failure modes: silent truncation, confident miscategorisation, and a lossy summary you cannot audit.
Why does it matter when you're choosing a tool?
Because memory is priced like inference and sold like storage, and the two do not reconcile. Rows in a memory store cost approximately nothing; the tokens it takes to read and write those rows are billed at the same rates as any other token. So the question is never "how much memory does the plan include" — it is "how many tokens does one turn cost with memory versus without it, and at what point does the transcript stop fitting".
Run the arithmetic on the vendors' own published figures (ours; rates are a mid-tier $3.00 per million input tokens, the cache-read rate is the 0.1× multiplier documented in prompt caching). The Mem0 paper puts a raw LOCOMO conversation at roughly 26,000 tokens and its own memory footprint at roughly 7,000.
| What you send each turn | Tokens | Effective input rate | Cost per turn |
|---|---|---|---|
| Full transcript, no caching | 26,000 | $3.00 / M | $0.078 |
| Full transcript as a cache read | 26,000 | $0.30 / M | $0.0078 |
| Retrieved memory, uncached | 7,000 | $3.00 / M | $0.021 |
| Retrieved memory as a cache read | 7,000 | $0.30 / M | $0.0021 |
Read the second row against the third. A warm cache makes replaying the entire transcript about 2.7× cheaper per turn than reading a 7,000-token memory. The 90%-fewer-tokens headline is real, and it is being compared against the one baseline — uncached full replay — that a competent team stopped shipping the day caching became available. If your sessions fit in the window and your cache stays warm, a memory layer is a new cost centre, not a saving.
Then add the write side, which the benchmark tables leave out. Extraction is not free: the Mem0 method calls an LLM at least twice per exchange — once to pull candidate facts, once to decide ADD/UPDATE/DELETE — and those calls are billed as normal input and output tokens. At 2,000 input and 300 output tokens per call on a mid-tier model, one exchange costs roughly $0.021 to write before it costs anything to read. Over 1,000 exchanges that is about $21 of writes against roughly $8 of cached replay for the same turns. Memory only starts winning when the alternative stops being available.
Which is the honest case for it, and it is a good one: memory is what you buy at the window boundary and across sessions. A support agent that must remember a customer from March in November cannot replay March; a coding agent that resumes after a context reset cannot replay what was truncated; a consumer product with a per-user profile cannot re-send every user's history on every request. In those cases the comparison is not memory versus cache — it is memory versus RAG over raw logs, or memory versus nothing. Against that baseline the 7k-token footprint is a bargain, and the 91% p95 latency reduction (1.44 s versus 17.12 s in the paper's LOCOMO runs) is the number that actually shows up in a product demo.
What is true in 2026 specifically?
What did Anthropic ship, and what does the "39%" measure?
On 2025-09-29 Anthropic shipped two features together: context editing, which automatically clears stale tool calls and results from the window as it approaches the limit, and the memory tool, which lets the model persist what it learned into files outside the window and read it back later. Anthropic's own announcement gives three measurements: combining both improved performance by 39% over baseline on its agentic evaluation; context editing alone delivered 29%; and on a 100-turn web-search evaluation, context editing let agents finish workflows that would otherwise fail on context exhaustion while cutting token consumption by 84%.
Read those three numbers in order and the product strategy is visible: most of the gain comes from deletion, not storage. Thirty of the thirty-nine points arrive from throwing away stale tool output. Memory is the smaller, more expensive partner — and it exists because once you delete, you need a place to have put things first. Anthropic's documentation is explicit that the memory tool is the "just-in-time" half of the pattern: store what you learn, retrieve on demand, keep the active context focused. It is also explicitly client-side, and eligible for zero-data-retention agreements — the model proposes, your infrastructure stores.
What does the memory layer actually cost per turn?
Two line items, and buyers usually price only one. The read is the retrieved memory in the prompt: 7,000 tokens in the paper's configuration, more if you concatenate several memory types. The write is the extraction pipeline, which runs whether or not anyone ever retrieves the result — a fixed cost per exchange, paid in full even for the turns that produced nothing worth remembering. On the published rates above, the write roughly matches the read for a mid-sized model, and dominates it for a cheap one. That is the opposite of a database, where writes are cheap and reads are the cost.
One more asymmetry worth naming: the memory footprint scales with entities, not with traffic. A store holding 7k tokens per user costs 7k tokens for user one and 7k for user ten thousand — linear in users, flat in queries. Compare that to a vector database, where the per-query meter scales with corpus size, and to KV cache, which scales with context length per request. Three different axes, three different bills; a stack can be cheap on one and expensive on the other two.
Why is the memory market's price list empty?
Check the vendors' own pages and the shape of the 2026 market is visible. Mem0's pricing page currently lists two options: Free — unlimited end users, 10,000 add requests and 1,000 retrievals per month for the first 30 days with model calls included, or 50,000 adds and 10,000 retrievals a month if you bring your own LLM key — and Custom, which is a contract. The self-serve paid tier in the middle is gone, and graph memory, user profiles and consolidation are gated behind the contract. Meanwhile the platform vendors have taken the other route: Anthropic sells the primitive (tool calls at token rates, storage on your side), and every agent framework ships its own session store for free.
That is what a category looks like when it is being absorbed into the platform layer. The honest 2026 recommendation follows from it: start with context editing plus caching, add a file-based memory only for state you must survive a reset, and buy an extracted-fact service only when you have a per-user profile that outlives the window and can measure the accuracy you are trading away. Which brings up the number least likely to appear in a vendor deck: in the Mem0 paper, the best LLM-as-a-Judge score belongs to full-context processing, with the graph variant at 68.44% and the base variant lower still. Memory does not beat reading everything. It beats reading everything on cost and latency, and it loses accuracy to do it. That is a fine trade — as long as you know you are making it.
Quick checklist
- Measure your session length against your window first. If p95 sessions fit, caching plus context editing beats a memory layer on cost — run the four-row table above with your real token counts before you evaluate any vendor.
- Price the writes, not the rows. Ask what the extraction pipeline calls per exchange, on which model, and whether it runs on every turn or only on session close. A memory product that extracts on every turn has a permanent, traffic-linear bill.
- Ask who decides what to forget. Replay truncates silently, a file-based tool lets the model decide in-band, an extraction service decides out-of-band with a prompt you cannot see. Pick the failure mode you can debug.
- Check where the bytes live. Client-side storage (Anthropic's model) keeps data in your infrastructure and can qualify for zero-retention terms; a managed memory service moves the same personal data into a third party. That is a compliance decision before it is a technical one.
- Treat "26% better" as relative. It is measured against a named competitor on one benchmark (LOCOMO), not against the ceiling — in the same paper, full-context scoring is higher. Budget for an accuracy drop and measure it on your own eval set.
- Do not buy memory to replace RAG. Documents belong in retrieval over a corpus; memory is for facts about the user and the task. Mixing them produces a store that is neither auditable nor cheap to query.
Where you'll hit it
Memory is the line item that appears once an assistant stops being stateless. In AI agent it is what makes multi-session work possible at all; in context window it is the answer to the window's two hard edges — length and cost; in prompt caching it is the alternative you should price against before buying it; in function calling it is why the memory interface is six tool commands rather than an API endpoint; and in fine-tuning vs RAG it is the third option neither side of that comparison covers — learned behaviour belongs in weights, facts belong in retrieval, and state belongs here. In the best AI customer-service tools ranking it is what "remembers your customers" actually runs on, and why per-resolution pricing hides a per-turn write. Full list on the AI tools glossary.
The bottom line
Agent memory is the persistence layer that lets an agent carry facts across sessions instead of replaying the transcript — and in 2026 it is three products wearing one name: replay session stores, a client-side file directory the model edits itself (Anthropic's six-command memory tool, shipped 2025-09-29 with context editing, where 29 of the 39 points of measured gain came from deleting stale context rather than storing new context), and extracted-fact services like Mem0 (26% relative over OpenAI's memory, 91% lower p95 latency, 90%+ fewer tokens than full replay, ~7k tokens of footprint against ~26k of transcript). Price it honestly: at a $3.00/M input rate, a warm cache makes replaying 26k tokens cost about $0.008 per turn, while reading 7k uncached memory tokens costs about $0.021 — plus roughly $0.021 per exchange to write them. Memory is not a token-saving purchase. It is what you buy when the transcript no longer fits, the session no longer persists, or the profile belongs to a user you must remember for a year.