What is mixture of experts?
Plain-English definition · Updated 2026-10-25. Parameter counts below are from official model cards and technical reports; they do not change with the weather, but new releases do.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
Mixture of experts (MoE) splits the model's feed-forward layers into many parallel experts and lets a small router send each token to only a few of them. The point is to grow a model's capacity without growing the work per token. The catch, for anyone choosing a tool, is that every MoE model therefore has two parameter counts. Total parameters is what you have to load into memory; active parameters is what actually runs for one token, and that is what sets speed and, indirectly, your API bill. DeepSeek-V3 is 671B total but 37B active. Mixtral 8x7B is 46.7B total but 12.9B active — which is also why it is not 56B, despite the name. Read a spec sheet with the wrong number and you will size hardware for a model that does not fit, or pay for speed you do not get.
What it actually means
A dense transformer runs every parameter for every token. A 70B dense model does 70B parameters' worth of arithmetic to emit one token, whether the token is "the" or a line of Python. MoE keeps the size and drops the obligation: inside each transformer layer, the feed-forward block is replaced by a set of separate expert networks — 8 in Mixtral, 256 routed experts plus one shared expert in DeepSeek-V3 — and a learned router picks the top few for each token. Mixtral picks 2 of 8. DeepSeek-V3 picks 8 of 256. The unselected experts stay loaded but idle.
Two things follow, and only one of them is in the marketing. The first is real: capacity per unit of compute goes up, because a token only pays for the experts it visits. Mistral's own launch post puts it plainly — Mixtral has 46.7B total parameters but only uses 12.9B per token, so it "processes input and generates output at the same speed and for the same cost as a 12.9B model." The second is the one that bites: all of the parameters still have to be resident. Nobody pages an expert in from disk mid-token at production latency. Mixtral's VRAM footprint is that of a 47B dense model, not a 13B one.
This is not a fringe architecture any more. DeepSeek-V3 — 671B total, 37B activated, 61 layers, trained on 14.8T tokens — made it the default way to build a frontier open-weights model, and the naming convention followed: Qwen3-235B-A22B carries both numbers in its name, 235B total, 22B active. When a model name contains an "A", that number is the active count, and it is the one that predicts how fast the thing responds.
Why it matters when you're picking a tool
Because the two counts point at two different purchase decisions, and vendors quote whichever one flatters the page. Total parameters is a memory number: it decides whether the model fits on your card, your node, or your budget at all. Active parameters is a throughput number: it decides how many tokens per second you get and how much compute the provider burns per call. An API sells you the second and absorbs the first. A self-hoster pays for the first and only gets the second's benefit if the arithmetic works out.
That arithmetic is unforgiving at the top of the range. DeepSeek-V3's 671B parameters, stored at one byte each — 8-bit, which is already compressed relative to the 16-bit weights most models ship in — is 671 GB before a single token of KV cache. There is no consumer configuration that holds it. An 8-GPU node with 141 GB per GPU (1.13 TB of HBM) can, in principle, hold the weights with room for cache; four 80 GB accelerators (320 GB) cannot, at any quantization that still resembles the model. The "37B active" line that makes the model feel like a 37B model is true about speed and irrelevant about fit.
The naming adds a second trap. Mixtral 8x7B is 46.7B parameters, not 56B — the experts replace only the feed-forward blocks, so attention, embeddings and the router are counted once rather than eight times. Multiply the label and you overestimate the memory by about a fifth; read only the active count and you underestimate it by nearly four times. Both errors are common, and both are avoidable by looking for the pair of numbers rather than the name.
One more assumption worth retiring: experts are not topic specialists. The intuitive story — a physics expert, a code expert, a poetry expert — is not what the routing analysis shows. Mistral's own paper reported that it found no obvious pattern in expert assignment by topic; selection tracked the syntax of the text more than its domain, especially in the first and last layers. "Expert" is a structural term, not a job title, which is why you cannot buy a model "with a coding expert" any more than you can buy a car with a specific cylinder.
The 2026 reality check
Sparsity moved from research trick to default, and the spec sheets got harder to read. Through 2024 the open-weights frontier was dense models in the 7B–70B band with MoE as an exception; by 2026 the largest open releases are almost all sparse, and several carry the active count in the product name precisely because buyers kept misreading it. If you are comparing a "235B" model against a "70B" dense model, you are comparing a 235 GB memory footprint against a 70 GB one — with compute closer to 22B against 70B. Those are not the same trade, in either direction.
The API price is the active count wearing a disguise. Per-token pricing on a hosted MoE reflects the compute the provider actually spends, which tracks active parameters far more than total. That is the honest reason a 671B model can be priced like a mid-size one — and the reason a "cheap per token" model can still be the expensive choice once you measure your own workload (see inference cost). The provider is renting you the active path and amortizing the total across many customers; you get none of that amortization when the box is yours.
Quantization changes the memory number, not the shape of the problem. Compressing weights to 4 bits cuts the resident footprint by roughly three-quarters against 16-bit, which is what makes 47B-class MoE models runnable on a 24 GB card at all — but it does not turn a 671B model into a laptop model. And because the router chooses per token, you cannot drop experts you think you will not need: the router decides, not you (see open weights for what you are allowed to do with the file once you have it).
Quick checklist
- Find both numbers before you compare anything. Total parameters sets memory, active parameters sets per-token compute. A spec sheet with one number is incomplete, and the missing one is usually the one that hurts.
- Size the hardware on the total, not the active count. Weights must be resident to serve at latency — Mixtral's VRAM need is that of a ~47B dense model even though each token runs 12.9B. Active parameters predict tokens per second, never gigabytes.
- Do the byte arithmetic rather than trusting the label. Total parameters × bytes per weight = weight memory. At 8 bits, 671B is ~671 GB; at 4 bits, ~336 GB; at 16 bits, ~1.34 TB. Add KV cache, which grows with context and batch (see context window).
- Never multiply the name. "8x7B" is 46.7B because attention and embeddings are shared. Check the model card for the pair of figures instead of doing algebra on the product name.
- Test the task, not the benchmark. A sparse model's routing is per token and per layer; behaviour on your specific prompts — especially long-context recall and tool calling — is not implied by its MMLU score. Run your own sample before switching.
Where you'll hit it
Three purchases on this site turn on the total-versus-active distinction: the best AI chatbots (where every frontier assistant you are ranking is almost certainly sparse, and the price gap between tiers is the active path, not the total size), ChatGPT vs Claude vs Gemini (three vendors, three unpublished architectures, one published price list — which is why we price behaviour rather than parameter counts), and Ollama vs LM Studio plus the best AI coding assistants (the two places where local MoE models actually get attempted, and where "it fits" means the total). Related terms: AI token is the unit the active count is spent in; inference cost is how that spend becomes an invoice; fine-tuning is where MoE complicates things further, since adapters usually attach to shared layers rather than to individual experts; and multimodal models are frequently sparse too, stacking a second axis of metering on top. Full list on the AI tools glossary.
The bottom line
Mixture of experts buys capacity with sparsity: split the feed-forward layers into experts, route each token to a few, and a model can be huge without being slow. That produces two parameter counts, and confusing them is the most expensive reading mistake in AI tooling — the total decides whether the model fits in memory (671B is 671 GB at 8 bits, whatever the active count says), the active count decides how fast and how cheaply each token comes out (37B for DeepSeek-V3, 12.9B for Mixtral 8x7B). Vendors quote whichever number flatters the page; read both, size hardware on the big one, and judge speed on the small one. And forget the story about topic specialists — the router follows syntax, not subject matter.