Home › Glossary

What is an embedding (and why do more dimensions not buy better answers in 2026)?

Plain-English definition · Updated 2026-11-06. Dimension menus, input limits, normalisation rules and prices come from OpenAI's model and embeddings documentation, Google's Gemini embeddings documentation and the Gemini Embedding general-availability announcement, Cohere's Embed documentation and the Amazon Bedrock model card for the same model. The arithmetic on storage and re-embedding cost is ours, and labelled as such.

AITop is an independent guide. Prices and features change fast — always check the vendor's page.

The 30-second answer

An embedding is a fixed-length list of numbers that represents a piece of text so that "close together" means "similar in meaning" — it is what lets software search by intent instead of by keyword, and it is the retrieval half of every RAG system. The number of dimensions is a design choice, not a quality score: Google's own published table for gemini-embedding-001 shows 1,536 dimensions at 68.17 on MTEB against 2,048 at 68.16, so a quarter-smaller vector measured no worse, and OpenAI states that text-embedding-3-large shortened to 256 dimensions still beats an unshortened text-embedding-ada-002 at 1,536. Prices are trivial next to what follows — $0.02 to $0.15 per million input tokens — because the bill arrives downstream: about 6 KB per vector of float32, a full re-embed of the corpus whenever you change models, and a silent quality loss if you embed documents and queries with the wrong mode.

What is an embedding, exactly?

An embedding is a fixed-length array of floating-point numbers, produced by a model, that positions a piece of content in a geometric space where distance means similarity. Feed the sentence "the invoice is overdue" to an embedding model and you get back a list — 1,536 numbers, say — and feeding it "this bill was not paid on time" produces a different list whose numbers are numerically close to the first. Nothing in either list is readable. What makes the list useful is that the model was trained so that the angle between two vectors tracks whether two humans would call the two texts the same thing.

That is the whole trick, and it has three consequences that decide real budgets. First, an embedding model is not a chat model: there is no output text, no prompt, no reasoning, and no per-token output charge — you pay for input tokens only, and the response is a block of numbers. Second, the vector must be stored somewhere that can search it, which is why a vector database appears in every architecture diagram; a million vectors is a data-engineering problem before it is a model problem. Third, and most often missed, the space is specific to the model that built it. Vectors from OpenAI's model and vectors from Google's model are lists of numbers with no shared coordinate system — you cannot mix them in one index, average them, or compare them, which is why swapping embedding models is a migration rather than a configuration change.

The mechanics of comparison are worth one paragraph because they explain the storage maths later. OpenAI states that its embeddings are normalised to length 1, so cosine similarity can be computed as a plain dot product and cosine and Euclidean distance produce the same ranking. Cohere accepts cosine, dot-product and Euclidean metrics. In practice all three vendors are selling you a unit-length direction plus a magnitude you can usually ignore, and every vector index you might use — HNSW, IVF, disk-based — is trading recall against the cost of holding those numbers in memory.

Why do three vendors disagree about how many dimensions you need?

Because "dimensions" is a storage decision dressed up as a model spec, and each vendor exposes the dial differently. OpenAI ships two sizes — text-embedding-3-small at 1,536 dimensions and text-embedding-3-large at 3,072 — with an 8,192-token input cap on both, and lets you shorten either at request time with a dimensions parameter. Its documentation is unusually direct about why that is safe: the v3 models were trained with a technique that lets developers "shorten embeddings ... without the embedding losing its concept-representing properties," and the worked example is that a 3-large embedding "can be shortened to a size of 256 while still outperforming an unshortened text-embedding-ada-002 embedding with a size of 1536." If you truncate vectors yourself rather than using the parameter, you must L2-normalise the result — OpenAI says so explicitly, and it is the step teams skip.

Google exposes a range instead of a menu: gemini-embedding-001 accepts anything from 128 to 3,072 dimensions and recommends 768, 1,536 or 3,072, over a 2,048-token input cap. Its documentation also states that when you use fewer than the full 3,072 dimensions on this model you must normalise the vector yourself. And Google publishes the table that should end the "more is better" argument: on MTEB, 2,048 dimensions score 68.16, 1,536 score 68.17, 768 score 67.99, 512 score 67.55, 256 score 66.19 and 128 score 63.31. Read those six numbers as a curve: between 512 and 2,048 the vector shrinks by 75% while the score moves by less than a tenth of a point. The cliff is below 256.

Cohere takes a third approach: embed-v4.0 offers 256, 512, 1,024 or 1,536 dimensions (1,536 by default) over a 128k-token context — the longest of the three by a factor of 16 against Google — and lets you choose the output number format, returning float, int8, uint8, binary or ubinary vectors. That last option matters more than it looks: a binary vector is 1 bit per dimension, which is 1/32nd the size of a float32 at the same dimension count.

All three are implementing the same idea, Matryoshka Representation Learning (MRL), published by Aditya Kusupati and colleagues at the University of Washington, Google Research and the Allen Institute for AI as arXiv:2205.13147 (May 2022) and presented at NeurIPS 2022. MRL trains the encoder so that the first m numbers of a vector are themselves a usable representation, which is why a prefix can be sliced off at request time. The paper's own headline result is up to 14× smaller embeddings at equivalent accuracy on ImageNet-1K. The property is trained in, not universal: naive truncation of a model that was not trained this way degrades quality. That is the difference between the dimensions parameter and cutting an array in half.

ModelDimensionsInput capList price / 1M input tokensThe catch
OpenAI text-embedding-3-small1,536 default, shrinkable via dimensions8,192 tokens$0.02Cheapest credible English default; half the vector size of 3-large at the same 8,192 cap
OpenAI text-embedding-3-large3,072 default, shrinkable8,192 tokens$0.136.5× the price of 3-small for a few MTEB points; worth it on hard, cross-lingual retrieval
Google gemini-embedding-001128–3,072, recommended 768/1,536/3,0722,048 tokens$0.15Shortest input window of the three — long documents must be chunked aggressively; must normalise below 3,072
Cohere embed-v4.0256/512/1,024/1,536, default 1,536128k tokens$0.12 (marketplace rate — see note)Text and image in one space; you must send input_type and get it right; ≤96 inputs per call

Price note: OpenAI's figures are the per-1M-token rates on its own model pages (2026-11-06). Google's $0.15 is the rate stated in its Gemini Embedding general-availability announcement. Cohere no longer publishes an Embed per-token rate on its pricing page, which now routes model access through enterprise and Model Vault plans; the $0.12 figure is the rate listed consistently by the AWS Bedrock, Azure AI Foundry and Oracle OCI marketplaces for cohere.embed-v4.0, and we have labelled it as a marketplace rate rather than a vendor page.

Why it matters when you're picking a tool

Because the embedding call is the cheapest line item in the stack, and everything downstream of it is not. Take a corpus of one million chunks at 500 tokens each: 500 million input tokens, which costs $10 at OpenAI's 3-small rate and $65 at 3-large. Now store them. A float32 vector at 1,536 dimensions is 6,144 bytes, so a million vectors is about 5.7 GiB of raw float payload before any index metadata — and an in-memory graph index typically holds more than the vectors themselves. The API call is rounding error; the RAM, the disk and the query latency are the procurement.

That arithmetic is also the honest case for shrinking the vector. Going from 1,536 to 768 dimensions halves storage and search work while Google's own numbers move the score from 68.17 to 67.99. Going to binary quantisation at 1,536 bits cuts the payload by 32×, to about 192 bytes per vector — 180 MB for the same million chunks — at a recall cost you should measure rather than assume. Cohere returning binary directly, rather than making you post-process, is a quiet signal that the industry expects most large indexes to end up quantised.

Because document mode and query mode are different calls. This is the bug that produces no error message. Both Cohere (input_type: search_document, search_query, classification, clustering) and Google (task_type: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, and so on) require you to declare what kind of text you are embedding, and the model applies a different prompt prefix per type accordingly. Embed your corpus with the query setting, or your queries with the document setting, and retrieval still returns results — just worse ones, with nothing in the logs. If a RAG demo feels disappointing and the plumbing looks correct, check this before you blame the model.

Because dimension is a schema decision you will live with. Vectors from two different models are not comparable, so changing embedding model means re-embedding every document — one million chunks is another $10 and a few hours, but 100 million chunks is a project with a freeze window. Teams that expect to migrate should store the raw text alongside the vector (you need it anyway for citations), keep chunking deterministic and versioned, and record which model and dimension produced each vector. A vector column with no provenance is a one-way door.

Because multilingual and multimodal change the choice more than quality does. If your corpus is English support tickets, 3-small at $0.02 is the rational default and the leaderboard gap does not survive contact with your own data. If it is scanned contracts, Cohere's 128k window and image support remove an OCR pipeline; if it is long policy documents, Google's 2,048-token cap forces chunking that will cost you more accuracy than any dimension choice. That is a capability question, and capability questions beat spec-sheet rankings every time.

The 2026 reality check

Embeddings went multimodal, and the input window split the market. Google's gemini-embedding-2 embeds images, audio and video into the same space as text — up to 8,192 total input tokens per request, with limits of six images, 180 seconds of audio and 120 seconds of video — so a page can be retrieved by a screenshot of it. Cohere's embed-v4.0 does text, images and interleaved page content over 128k tokens, and publishes its own tokenisation rule for images: roughly (pixels ÷ 784) × 4 tokens. Meanwhile the text-only models that win on MTEB remain deliberately narrow: 2,048 tokens on gemini-embedding-001 is 2.5 pages. The practical consequence is that "which embedding model" is now two questions — how long is my document, and is any of it a picture.

The benchmark numbers are not comparable, and the vendors know it. OpenAI reports MTEB (English) figures — 62.3 for 3-small, 64.6 for 3-large — while Google reports MTEB Multilingual v2 at 68.17 and publishes its own dimension-vs-score table. Different task sets, different languages, different normalisation. Quoting "68 beats 64" across those two columns is not an analysis, it is a category error, and it is the single most common mistake in embedding-model comparison articles published in 2026. The only number that decides your deployment is your own labelled retrieval set, run on your own chunks.

The dimensions arms race quietly reversed. The 2023 story was 1,536 → 3,072. The 2026 story is that three vendors now let you go down — OpenAI via dimensions, Google across a 128–3,072 range, Cohere across four fixed sizes — because retrieval quality turned out to saturate well below the maximum and index cost did not. A team buying a vector database on the assumption that vectors will get bigger is planning for the last generation. Budget for fewer, cheaper dimensions and for the storage savings funding a reranker, which usually buys more accuracy per dollar than the extra 1,536 floats did.

What has not changed: nothing here makes retrieval correct. An embedding measures similarity of phrasing, not truth or relevance — the same limitation that makes RAG fail on questions whose answer is not in the corpus, and that keeps hallucination a separate entry in this glossary. If the top-10 chunks do not contain the answer, no dimension count will invent it.

Quick checklist

Where you'll hit it

Every retrieval-backed product on this site sits on top of an embedding index, whether or not the landing page says so. It shows up hardest in three places. Support and knowledge tools — a customer service assistant that cites the wrong help article is almost always a chunking or query-mode problem, not a model problem. Code search and repo agents, where the corpus is code and the useful chunk is a function rather than a paragraph — see what that does to coding assistants and to Claude Code versus Cursor. And local RAG stacks, where the embedding model runs on your own GPU and the dimension you pick is VRAM you cannot get back — the same arithmetic that governs quantisation and the KV cache in tools compared in Ollama versus LM Studio. Related entries worth reading alongside this one: RAG for what happens after retrieval, context window for the budget the retrieved chunks compete for, and token pricing for the arithmetic that makes the embedding call the cheapest line in the stack. Full list on the AI tools glossary.

The bottom line

An embedding is the cheapest and most consequential decision in an AI product: a few dollars of API calls that determine whether your search works and whether you can ever change your mind. The marketing frame — "more dimensions, better model" — is the wrong axis, and the vendors themselves have stopped pretending otherwise: Google publishes a table where 1,536 dimensions score marginally above 2,048, OpenAI ships a parameter whose documented purpose is to make vectors smaller, and Cohere lets you ask for one bit per dimension. What you are actually choosing is three things: an input window (2,048 tokens or 128k — this decides your chunking), a vector size (which decides your RAM bill and your query latency), and a vector space (which decides whether you can ever migrate without re-embedding everything). Pick the cheapest dimension that passes your own retrieval test, keep the source text and the model name beside every vector, and check your document/query mode before you blame the model. The numbers are cheap; the schema is the commitment.