What are structured outputs (and what do they still not guarantee in 2026)?
Plain-English definition · Updated 2026-11-05. Parameter names, schema limits and refusal behaviour come from OpenAI's, Google's and Anthropic's own API references — plus Microsoft's Azure OpenAI documentation, which publishes the operational limits the OpenAI page leaves implicit. The reading of what they add up to is ours.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
Structured outputs is a decoding mode that constrains every token the model generates against a JSON Schema you send with the request, so the answer is guaranteed to parse and to carry exactly the field names, types and nesting you declared. OpenAI calls it "the evolution of JSON mode" and states plainly that "only Structured Outputs ensure schema adherence." Google's docs are blunter about the cheap version: setting a JSON MIME type without a schema "acts only as a strong hint to the model and doesn't ensure 100% valid JSON." Anthropic shipped the same idea, reached general availability on 4 February 2026, and enforces hard complexity budgets — 20 strict tools, 24 optional parameters, 16 union-typed parameters per request, with a 180-second schema compilation timeout. The catch that decides purchases: the guarantee covers shape, not truth. A schema-perfect object can hold a fabricated figure, and the three vendors accept different subsets of JSON Schema — a schema that runs cleanly on one provider returns HTTP 400 on the next.
What structured outputs actually guarantee
Normally a language model picks the next token from a probability distribution over its whole vocabulary. Structured outputs inserts a filter: at each step, tokens that would break your schema are masked out before sampling. The result is not a better prompt — it is a grammar the decoder cannot violate. That is why the guarantee is unusually strong and unusually narrow at the same time. It is strong because it is mechanical: if your schema says priority is one of three strings, no run will ever return a fourth. It is narrow because a grammar has no opinion about content. A field called amount will always be a number and can always be the wrong number.
OpenAI draws that line explicitly, and the distinction is the single most useful sentence in any of the three docs: "Structured Outputs is the evolution of JSON mode. While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence." The older json_object mode guarantees parseable JSON and nothing else — no required keys, no types, no nesting. OpenAI's recommendation is unambiguous: "We recommend always using Structured Outputs instead of JSON mode when possible." It is enabled by sending a schema with strict: true, either as text.format in the Responses API or as response_format in Chat Completions, and it is available from the gpt-4o-2024-08-06 and gpt-4o-mini snapshots onward.
The guarantee has three documented escape hatches, and every production system eventually meets one. Safety refusals: OpenAI adds a field outside your schema because "a refusal does not necessarily follow the schema you have supplied in response_format," so the response carries a refusal property instead of your object. Truncation: if the answer hits the token ceiling, the JSON ends mid-object — OpenAI's own sample code checks incomplete_details.reason === "max_output_tokens" before parsing. Case drift: Anthropic warns that structured outputs "don't guarantee the capitalization of string enum and const values." None of these is a bug. They are the reasons a schema is a contract about format, not a promise of a well-formed application.
Why does a schema work on one provider and fail on another?
Because "supports JSON Schema" means three different things. Each vendor accepts a subset, and the subsets do not nest. OpenAI's supported types are string, number, boolean, integer, object, array, enum and anyOf, with recursion via $ref allowed. Google supports anyOf, format, items, minItems/maxItems, minimum/maximum, nullable and propertyOrdering — but only string enums. Anthropic supports string formats including date-time, email, uuid and ipv4, and explicitly does not support recursive schemas, numeric constraints such as minimum/maximum/multipleOf, or string length limits. Write a schema against the union of all three and you will get a 400 from at least two of them.
The subtler trap is what "optional" means. OpenAI requires every property to appear in required; a genuinely optional field is emulated with a union type that includes null, and every object needs additionalProperties: false. Google is the reverse — "By default, fields are optional, meaning the model can populate the fields or skip them" — so a field you forgot to mark required will sometimes simply be absent. Anthropic allows optional parameters but prices them: they count against a shared budget of 24 optional parameters across all strict schemas in one request, because, in its words, "each optional parameter roughly doubles a portion of the grammar's state space." The same design decision is a mandatory null-union on one platform, a silent omission risk on the second, and a scarce quota on the third.
| Provider | How you turn it on | Optional fields | The limit you'll hit |
|---|---|---|---|
| OpenAI | strict: true on a json_schema format | Not allowed — every field is required; use type: ["string","null"] | Microsoft's Azure OpenAI doc for the same API publishes 100 object properties and 5 nesting levels; no minLength, pattern or format |
| Google Gemini | responseSchema plus application/json MIME type | Optional unless marked required | String enums only; a complex schema returns InvalidArgument: 400; schema tokens bill as input |
| Anthropic Claude | output_config.format, or strict: true on tools | Allowed, but 24 total per request | 20 strict tools, 24 optional params, 16 union params, 180 s compile timeout → "Schema is too complex for compilation" |
Why it matters when you're picking a tool
Because it moves your integration risk from the model to the schema. Before structured outputs, teams shipped retry loops and regex repairs; now they ship schemas, and a schema is portable in theory and fragile in practice. If your product extracts invoices, tickets or lab results, the schema is the part of your codebase most likely to break on a model swap — which makes it the part you should be able to test across providers before you sign. This is the same failure mode as the temperature ranges that disagree by a factor of two between vendors: the parameter looks universal and is not.
Because the schema is billed. Google states it outright: "The size of your response schema counts towards the input token limit." Anthropic says your "input token count is slightly higher." On a long-context pipeline where the schema is fixed and the documents are large, that is noise; on a high-volume classification job with a 300-line schema and a 20-token input, the schema is most of what you pay for. Run the arithmetic with token pricing before you assume the structured version of a call is free.
Because the first call is slow and the cache is fragile. OpenAI notes that "the first request you make with any schema will have additional latency as our API processes the schema," and Anthropic adds the operational detail that compiled grammars are cached for 24 hours from last use and invalidated when the schema structure or the tool set changes — but not when you only edit a name or description. A team that ships a schema change every deploy keeps paying a cold grammar, and anyone who changes the output format also "invalidate[s] any prompt cache for that conversation thread." Two caching systems, two invalidation rules, one bill.
Because "it's structured, so it's reliable" is how bad data reaches production. Constrained decoding fixes the parse, not the number. Extraction tools in categories like AI customer service and AI coding assistants are exactly where this bites: a ticket classifier that always returns one of four labels will always look healthy on a schema check while quietly mislabelling a fifth of traffic. Schema validation replaces your JSON parser; it does not replace your evaluation set.
The 2026 reality check
The feature is now table stakes, and the differentiation moved to the limits. OpenAI has had it since the August 2024 model snapshots. Anthropic ran it as a public beta from November 2025 (the old beta header was literally named structured-outputs-2025-11-13) and moved to general availability on 4 February 2026, at which point the parameter moved from output_format to output_config.format and the beta header became unnecessary — using the old parameter without the header now returns a 400. That is a migration event, not a footnote: any code pinned to the beta shape breaks silently on upgrade.
The real 2026 differentiator is what happens when the schema is too big. Anthropic publishes the only explicit complexity budget in the market — 20 strict tools, 24 optional parameters, 16 union-typed parameters, a 180-second compilation timeout — and notes the failures are counter-intuitive: "if you have 4 strict tools with 6 optional parameters each, you'll reach the 24-parameter limit even though no single tool seems complex." Google takes the other route, letting a complex schema through the front door and returning InvalidArgument: 400 with advice to "shorten property names," "flatten nested arrays" and "reduce the number of valid values for enums." OpenAI simply refuses the schema's shape up front. Three vendors, three failure modes, one shared lesson: schema design is now a capacity-planning exercise, not a formatting preference.
And the honest framing of what changed. Two years ago the problem was that models produced prose when you wanted JSON. That problem is solved — the guarantees are real and they are cheap to obtain. What replaced it is the older problem underneath: a well-formed answer can be wrong. Structured outputs did not make AI outputs trustworthy; it removed one specific class of failure and left the rest exactly where it was, which is why hallucination is still a separate entry in this glossary and still a separate line in every serious evaluation. Anthropic's own caveat — that a refusal "takes precedence over schema constraints" — is the polite version of the same warning.
Quick checklist
- Never ship
json_object/ bare JSON mode as a reliability feature. OpenAI says only Structured Outputs ensure schema adherence; Google says the MIME type alone "doesn't ensure 100% valid JSON." If the field matters, the schema must be present. - Test the schema on every provider you might migrate to. All-required objects,
additionalProperties: falseand nominLength/patternis the portable subset. Anything richer is a vendor commitment, not a JSON Schema commitment. - Treat optional fields as a budget, not a style. Anthropic caps you at 24 optional parameters per request; OpenAI does not allow them at all and wants nullable unions; Google will silently omit them. Pick per provider, do not copy.
- Budget the schema on the input bill. Google bills schema tokens as input; Anthropic raises the input count. On high-volume, short-prompt jobs, measure it before assuming it's negligible.
- Write the refusal and truncation path first. Check the
refusalfield (OpenAI),stop_reason: "refusal"(Anthropic, returned with HTTP 200) andmax_tokenstruncation before you parse. A schema does not cover these. - Keep the eval set. Schema validity is a unit test. A fourth grader can pass it while the content is wrong — see RAG for the retrieval half of the same problem.
Where you'll hit it
Every tool comparison on this site quietly assumes structured outputs now that the answer has to land in a table: a ChatGPT vs Claude benchmark scored by an automated grader is only as honest as the grader's schema, and a Claude Code vs Cursor comparison depends on both agents emitting tool arguments that parse. It shows up hardest in three places. Extraction pipelines — invoices, tickets, contracts — where the schema is the product. Agent loops, where an agent's tool call is structured output with strict: true on top, and one malformed argument ends the run. And evaluation harnesses, where the judge model's output format decides whether your benchmark numbers mean anything at all. Related terms worth reading alongside this one: rate limits for the ceiling that ends a structured pipeline before the schema does, reasoning tokens for the output that counts against your max_tokens before the JSON starts, and context window for the budget the document and the schema share. Full list on the AI tools glossary.
The bottom line
Structured outputs is a real engineering win sold with a misleading frame. The mechanism is honest — a grammar constrains decoding, the JSON parses, the field names arrive as declared — and it retires an entire category of integration bug. What it does not do is make the content true, and the marketing shorthand ("guaranteed output") invites exactly that confusion. For a buyer, the useful way to read the feature is as a portability cost: three vendors, three JSON Schema subsets, three different meanings for "optional," and three different failure modes when a schema gets big — 5 nesting levels on the OpenAI-flavoured API, a 400 on Gemini, a 180-second compile timeout and a 24-optional-parameter ceiling on Claude. Write to the smallest common subset, budget the schema against your input tokens, always implement the refusal and truncation path, and keep an evaluation set that can catch a well-formed lie. The JSON was never the hard part.