Home › Glossary

What is distillation (in plain English)?

Plain-English definition · Updated 2026-11-14. Sources are the vendors' own documentation (OpenAI's supervised fine-tuning guide and deprecation timeline, Google Cloud's Gemini distillation service page, Anthropic's glossary) plus DeepSeek's January 2025 R1 report (arXiv 2501.12948) and the model card figures it published. Distillation availability is changing quarter by quarter; the structural question in this entry — who owns the student — has not changed.

AITop is an independent guide. Prices and features change fast — always check the vendor's page.

The 30-second answer

Distillation is training a small model on a big model's outputs so the small one copies the big one's behaviour on your task. OpenAI's fine-tuning guide states the goal in one line: "This technique can enable you to train a small model to perform similarly on a specific task to a larger, more costly model." It works — DeepSeek's R1-Distill-Qwen-32B scored 72.6% on AIME 2024 against OpenAI o1-mini's 63.6%, on a model small enough to run on one workstation. But the 2026 buyer's question is not "does it work," it is who owns the student: OpenAI's guide now carries the line "OpenAI is winding down the fine-tuning platform," Google's distillation service is allow-listed preview only (teacher gemini-3.1-pro → student gemini-2.5-flash, us-central1), and Anthropic's API offers no fine-tuning at all, so no distillation either. On a platform that is closing, a distilled model has an expiry date written by someone else's roadmap — eligibility for new training jobs runs to 2027-01-06, and the students stop when their base models do.

What does distillation actually mean?

Distillation is a procedure with four steps, and the order matters. First you make the teacher good at your task — prompt engineering and evals, not training. Then you capture the teacher's answers on a large set of your own inputs. Then you filter those answers against your evaluation criteria to build a dataset. Finally you fine-tune a small model on that dataset. OpenAI's guide describes exactly this loop and puts the sequencing in bold: "Good evals first!" — "only invest in fine-tuning after setting up evals," because without a reliable way to tell whether the student beats the base model, you are paying to move a number you cannot see.

The definition worth memorising: distillation transfers behaviour, not knowledge. The student never sees your database or the teacher's training corpus; it sees a few thousand (prompt, answer) pairs and learns the mapping. That is why the same technique that fixes tone, format and instruction-following does almost nothing for facts — a point that decides the build order: RAG for what the model should know, distillation for how it should behave, and fine-tuning as the general label for that second half. OpenAI's guide recommends starting at 50–100 examples and says that "if 50 examples have no impact, rethink your task or prompt before adding training data" — a useful brake on the intuition that more data fixes a badly framed task.

Google's version of the definition adds one clause that matters technically. On the Gemini distillation service page, Google distinguishes distillation from plain supervised fine-tuning by what it copies: not only the teacher's final text output but the teacher's internal reasoning paths. That is why Google positions the service for tasks where "the base Flash model struggles but the Pro model succeeds" — multi-step logic, technical summarisation, complex coding — and why the input to a distillation job can be a prompt-only dataset: you supply the questions, the teacher supplies both the reasoning and the answer. OpenAI's step-by-step guide, by contrast, describes capturing final responses through the API (and notes that "the Responses API stores model responses for 30 days by default"), which means the data you distil from is whatever the teacher chose to say out loud.

Why does it matter when you're choosing a tool?

Because it is the only mechanism that changes the price per token of your own workload without changing the model you depend on. Take OpenAI's published list prices: gpt-4.1 bills $2.00 per million input tokens and $8.00 per million output tokens; gpt-4.1-mini bills $0.40 and $1.60. That is a 5× gap between teacher and student. Training on that platform costs $25.00 per million tokens for gpt-4.1 and $5.00 for gpt-4.1-mini.

Run the break-even (ours). One million teacher output tokens at $8.00 builds a dataset; training a million tokens on the mini rate costs $5.00; total $13.00 to stand up the student. Every million output tokens the student then serves saves $8.00 − $1.60 = $6.40. Break-even lands at 2.03 million student output tokens — about two weeks of a modest support bot, and pure margin after that. That arithmetic is the entire commercial case for distillation, and it is why the technique survived even as fine-tuning platforms consolidated: the saving is not a discount, it is a different cost structure.

The buyer-facing consequence is a question you can put to any vendor that claims a "custom small model": which teacher, and under what terms? If the answer is "we distilled a frontier model through its API," three follow-ups decide whether the product has a future — does the provider's terms of service permit training a competing model on its outputs, does the student's host platform allow the weights to leave, and what happens to your model when the teacher is deprecated. A distilled model is a derivative instrument: its value is real, and its life is somebody else's decision.

What is true in 2026 specifically?

Does OpenAI still let you distil?

For existing eligible customers, on a clock. The fine-tuning guide now states: "OpenAI is winding down the fine-tuning platform. The platform is no longer accessible to new users." Eligible existing customers "can create training jobs until January 6, 2027, unless the base model shuts down earlier," and fine-tuned models "remain available for inference until their base model shuts down." The published shutdown dates are concrete: gpt-4.1-nano-2025-04-14 and its fine-tuned versions on 2026-10-23; gpt-4.1-2025-04-14 and gpt-4.1-mini-2025-04-14 with their fine-tuned versions on 2027-01-14. Read those two dates next to each other and the strategic reading is unavoidable: the distillation recipe is still documented, and the door closes on 2027-01-06, eight days before the last student is switched off. Anyone planning a 2027 pipeline on OpenAI distillation is planning into a wall.

What is Google shipping, and what is the catch?

Google Cloud publishes a native Gemini distillation service with named teacher and student: gemini-3.1-pro teaching gemini-2.5-flash, during a preview that requires your project to be added to an allow-list, with the API enabled and jobs run in us-central1 only. The stated outcomes are the ones that matter commercially — lower latency and lower cost at "Pro-level reasoning" for a narrow task, plus the case where you have plenty of prompts but no labelled answers, which is the normal state of an enterprise with logs and no annotation budget. The catch is not hidden, it is structural: a preview with an allow-list, a single region, one teacher/student pair, and the student living inside the same platform that trained it. That last point is the lock-in, and it is the honest trade — you get a turnkey pipeline, and you get it where the teacher lives.

What about Anthropic, and the open-weight route?

Anthropic's own glossary states that the Claude API "does not currently offer fine-tuning." No fine-tuning, no supervised distillation — the same door, closed, for the same reason: a vendor that will not let you train on your data will not let you train on its outputs either. The open-weight route is the opposite extreme, and DeepSeek's R1 release is its best-documented case. The R1 report describes 800,000 distilled samples used to fine-tune six dense students — Qwen-based 1.5B, 7B, 14B, 32B and Llama-based 8B, 70B — for 2–3 epochs at a 32,768-token context. The headline result: R1-Distill-Qwen-32B scored 72.6% pass@1 on AIME 2024 against o1-mini's 63.6%, and 94.3% on MATH-500 against 90.0%. Note where the student still lost — Codeforces rating 1,691 vs 1,820 — because that is what distillation looks like up close: a large gain in the behaviour the teacher demonstrated most, and no gain where the teacher's traces were thin.

Two things about that release are routinely skipped. First, DeepSeek reported that distilling from the strong teacher beat running large-scale reinforcement learning directly on the same small base — the student inherits a distribution it could not have discovered on its own, which is also the reason it cannot exceed its teacher. Second, the licence follows the base architecture: the R1 weights are released permissively, while DeepSeek's release notes say the distilled models remain subject to their respective Qwen and Llama base-model licences. The student is yours to run; the terms are inherited.

Does distillation replace RAG or quantization?

No, and confusing the three is the most expensive mistake in this corner of the stack. Quantization shrinks the weights of a model you already have — same behaviour, fewer bits, one formula. RAG changes what the model can look up — it adds facts without touching weights. Distillation changes which model you run: it buys behaviour at a fifth of the token price, and it pays for that with a training pipeline, a dataset you must curate, and a dependency on a teacher you do not control. Enterprises that distil first and curate data second usually end up with a cheap model that is confidently wrong at scale.

Quick checklist

Where you'll hit it

Distillation is the hidden third option in every "should we build or buy" conversation on this site. In fine-tuning vs RAG it is the reason the fine-tuning half of the answer keeps shrinking — the training step it depends on is closing at one vendor and preview-only at another. In inference cost it is the only line item that moves the price per token rather than the number of tokens. In open weights it is why an MIT-licensed teacher produces students that are not MIT-licensed. In the best AI customer-service tools ranking it is what "custom model trained on your tickets" usually means — and why that claim deserves the three follow-up questions above. Related terms: fine-tuning is the umbrella procedure, quantization is the cheap alternative that changes bytes instead of behaviour, and AI hallucination is what a distilled model does when it inherits confidence without coverage. Full list on the AI tools glossary.

The bottom line

Distillation is training a small model on a large model's outputs so the small one behaves like the big one on your task — a 5× token-price gap on OpenAI's own list, paid for once with a dataset. In 2026 the technique is proven (DeepSeek's 32B student beat o1-mini on AIME 2024, 72.6% to 63.6%) and the supply is closing: OpenAI's fine-tuning platform is shut to new users with the last training job on 2027-01-06, Google's distillation service is allow-listed preview only with one teacher-student pair, and Anthropic offers no fine-tuning at all. So the buying question is not "can we distil" but "who owns the student, and until when" — and the answer is usually the vendor's roadmap, not yours.