What is function calling (and why does every request pay for it)?
Plain-English definition · Updated 2026-11-08. The mechanics, billing rules, schema limits and mode tables come from OpenAI's function calling guide (developers.openai.com), Anthropic's tool use documentation (platform.claude.com), and Google Cloud's Vertex AI function calling documentation (docs.cloud.google.com) — three first-party sources, each cited for its own vendor. The per-request token arithmetic is ours, and labelled as such.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
Function calling (the same feature Google's docs call tool use) is the mechanism that lets a model nominate an action instead of writing prose: it returns a tool name plus JSON arguments that match a schema you declared, and your application runs the actual function and feeds the result back. The model never executes anything — it issues structured requests, and the five-step loop is request → tool call → your code runs → tool output → final answer. Three facts from the vendors' own documentation matter when you buy: ① definitions are billed as input tokens on every request — OpenAI's docs state that "functions are injected into the system message … and are billed as input tokens," and Anthropic's docs put the floor for merely enabling tools at 286–675 tokens of system prompt before your definitions count; ② schema adherence is now guaranteed by all three majors (strict mode on OpenAI and Anthropic, ANY/VALIDATED modes on Google) but only for the shape of the arguments — Anthropic's own docs concede the model "might also infer a reasonable value" for a missing parameter; ③ forced tool calls are being retired — Anthropic's newest models return a 400 error for forced-call modes, and forcing a call skips the model's thinking entirely.
What is function calling, exactly?
Function calling is a structured handshake between a model and your code. Google's Vertex AI documentation defines the term directly: "Function calling, also known as tool use, provides the LLM with definitions of external tools … the model intelligently determines if a tool is needed and, if so, outputs structured data specifying the tool to call and its parameters." You declare tools up front — a name, a human-language description of when to use it, and a JSON Schema of the arguments — and the model responds with the name plus arguments instead of (or alongside) prose. Your application executes the real function, returns the output, and the model writes the final answer. OpenAI's guide spells out the five steps: make a request with tools, receive a tool call, execute code on your side, send the tool output back, and receive the final response (or more tool calls).
The definition is where vendors diverge in the small print, and the small print is the product. Anthropic restricts tool names to the regex ^[a-zA-Z0-9_-]{1,128}$ and its docs insist on descriptions of "at least 3–4 sentences," calling detailed descriptions "by far the most important factor in tool performance." Google's schema support covers exactly eleven OpenAPI attributes — type, nullable, required, format, description, properties, items, enum, anyOf, $ref, $defs — with everything else rejected: "Remaining attributes are not supported." OpenAI's strict flag trades flexibility for guarantees: it requires additionalProperties: false on every object and every field marked required, with optionality simulated by allowing null as a type.
Function calling is the engine under everything called an AI agent: the loop that lets a chatbot check the weather, a coding assistant edit a file, or a support bot read your order history. The intelligence is the model's; the capability is entirely the tool surface you expose — which is why the number of tools, and the tokens they cost, are billing decisions.
Why do tool definitions show up on your bill?
Because the definitions travel inside every request, and the vendors say so in writing. OpenAI's documentation is explicit about the mechanism: "Under the hood, functions are injected into the system message in a syntax the model has been trained on. This means callable function definitions count against the model's context limit and are billed as input tokens." Anthropic's documentation quantifies the floor: with tools present, the API appends a system-prompt block of 286–675 tokens depending on the model (496 tokens for Claude Sonnet 4.5 in auto mode, 588 in forced mode, for example) — before a single one of your own tool definitions is counted. Google bills the declarations the same way, as part of the input.
The arithmetic (ours, not the vendors'): a twenty-tool agent pays thousands of input tokens per request for the privilege of having tools. Take OpenAI's own soft suggestion — "aim for fewer than 20 functions available at the start of a turn" — with Anthropic's description guidance of 3–4 sentences plus a modest parameter schema: about 150–400 tokens per tool definition. Twenty tools therefore ship roughly 3,000–8,000 input tokens per request, plus Anthropic's 286–675-token floor. Now multiply by the request count: an agent that runs ten internal turns per task turns that into ten payments per task, and tool-heavy MCP setups with dozens of servers multiply it further. This is the mechanical reason OpenAI added tool search (deferred tools loaded only when needed, on gpt-5.4 and later) and recommends "limiting the number of functions loaded up front" — the definitions are a meter, not a configuration file.
The mitigation everyone runs is prompt caching: definitions sit at the front of the prompt as a stable prefix, so the cached-read discount applies on subsequent requests — OpenAI's allowed-tools mode is explicitly paired with prompt-caching optimisation in its docs. The un-mitigated case is the one to budget for: a new conversation, an evicted cache, or a schema you edit daily (cached prefixes break when the prefix changes) all reset the tax to full price.
The 2026 reality check
Schema adherence has converged in vocabulary and diverged in mechanics. All three majors now promise the arguments will match your schema — OpenAI's docs say strict: true "will ensure function calls reliably adhere to the function schema, instead of being best effort. We recommend always enabling strict mode"; Anthropic's docs say strict tool use ensures "Claude's tool calls always match your schema exactly"; Google's docs say the ANY and VALIDATED modes "ensure function schema adherence." But the guarantees are engineered differently: OpenAI demands all-required fields and forbids additionalProperties; Google accepts only eleven schema attributes and notes that "Starting with Gemini 3, this mode also enforces the presence of required parameters"; Anthropic applies strictness to custom tools with its own complexity quotas (the ones we tabulated in the structured outputs entry). Same sentence, three different engines — porting an agent between providers is a translation job, not a flag swap.
Guaranteed shape is not guaranteed truth. Anthropic's own documentation demonstrates the limit: for a required location parameter when the user says only "What's the weather?", Claude "might also infer a reasonable value" — inventing "New York, NY" — and the docs add: "This behavior is not guaranteed, especially for more ambiguous prompts and for less capable models." Strict mode validates the arguments against the schema; it cannot know the city is wrong. This is the same conclusion the structured outputs entry reached for JSON responses, now proven for tool arguments: the schema constrains form, and hallucination walks straight through it wearing valid JSON.
Forced tool calls are being retired by the vendors themselves. Anthropic's docs state that on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1, the forced-call modes any and tool "return 400 error," with the recommended alternatives being auto mode plus strict tool use, or structured outputs. The same docs note that "forcing a tool also skips thinking on every model that accepts forced tool use: the response starts with the tool_use block and has no thinking block." The design pattern the vendors are pushing is: let the model decide (auto/AUTO is the default everywhere), make the schemas strict, and validate outputs yourself. If your integration still hard-requires a call, plan for it to be a legacy behaviour.
Parallel calls are default-on everywhere, and each vendor has a different off-switch. OpenAI: "The model may choose to call multiple functions in a single turn. You can prevent this by setting parallel_tool_calls to false." Anthropic: a disable_parallel_tool_use flag nested inside tool_choice. Google: no opt-out flag at all — instead a return contract, "return all corresponding functionResponse parts together in a single Content object," matched by id (Gemini 3 and later) or by name and order, with a name mismatch failing as a 400 error. One more footgun from OpenAI's docs: for fine-tuned models, "if the model calls multiple functions in one turn then strict mode will be disabled for those calls."
Quick checklist
- Count your tool tokens before you count your users. Definitions × requests is a real bill line: ~20 tools with detailed descriptions ≈ 3,000–8,000 input tokens per request, paid on every turn. If that number embarrasses you, use deferred tools / tool search (OpenAI, gpt-5.4+) or trim descriptions.
- Put caching behind your definitions — and stop editing them. A stable tool prefix is prompt-caching gold; a schema you tweak daily resets the discount. Ship tool versions like code, not like notes.
- Design for auto, not forced. Anthropic's newest models reject
any/toolwith a 400, and forced calls skip thinking. If a workflow genuinely requires a call, enforce it in your application logic after an auto-mode response, not with a deprecated flag. - Validate values, not just schemas. Strict mode guarantees the JSON shape; Anthropic's docs show the model inventing plausible parameter values. Range-check arguments before executing — a valid-schema wrong-city lookup is a real support ticket.
- Assume parallel calls in your response parser. All three vendors return zero, one or several calls per turn; OpenAI's docs say "it is best practice to assume there are several." A parser that handles one call breaks silently the day the model batches two.
- Port between providers with a mapping table.
required/any/ANYare three dialects of "force a call"; the off-switches live in three different places. Budget the translation as engineering work, not a find-and-replace.
Where you'll hit it
Function calling is the plumbing of every product on this site that does something rather than just says something. Coding assistants are function calling with a file system attached — AI coding assistants and the tools compared in Claude Code versus Cursor live or die on how their tools are described and batched, because a mis-described tool is an edit to the wrong file. Chatbots with search — the assistants ranked in best AI chatbots and compared in ChatGPT versus Claude all decide per turn whether to call a web-search tool, which is why the same question sometimes gets a cited answer and sometimes a memorised one. Support and automation tools — the platforms in AI customer service tools expose order lookups and refunds as tools, which makes tool-surface cost (and the per-request token tax) a pricing input for their per-resolution invoices. Related entries: AI agent for the loop this enables, structured outputs for the sibling guarantee on JSON responses, prompt caching for the discount that softens the definition tax, and token pricing for what an input token costs. Full list on the AI tools glossary.
The bottom line
Function calling is how a model asks your code to do something — and 2026's documentation, read closely, teaches one economic lesson and one design lesson. The economic one: tool definitions are billed as input tokens on every request (OpenAI states it; Anthropic quantifies the floor at 286–675 tokens), so a wide tool surface is a per-request tax that only caching and deferred tool loading soften — size your tool surface like a subscription, not a setting. The design one: schema adherence is now guaranteed by all three majors, forced calls are being deprecated out (Anthropic's newest models 400 on them, and they skip thinking anyway), and parallel calls are default-on with three different off-switches — so build agents that run in auto mode with strict schemas, parse for multiple calls, and validate the values yourself, because the vendors guarantee the shape of the arguments and openly document the model inventing them. The tools are the capability; the definitions are the bill; the schema is a form, not a truth guarantee.