Inference provider vs GPU cloud: buy tokens, or rent the GPUs behind them (2026)
Data last updated: 2026-11-15. Pricing may have changed — verify with the vendor.
AITop is an independent guide. Prices and features change fast — always check the vendor's page.
New to the jargon? AITop's AI glossary explains it in plain English.
Related: what inference actually costs · why tokens per second is a throughput claim · what open-weights means for hosting · the 50% discount for patient workloads
An inference provider sells you tokens over an API — you pay per million, and pay nothing between requests. A GPU cloud rents you the accelerators themselves — you pay per GPU-hour, whether or not the card is busy. The same open-weights model usually sits on both rails, so the choice is not about quality; it is about your load shape. In November 2026 the two markets are both liquid enough to price precisely: DeepSeek-V3 output ranges from $0.20 to $1.50 per 1M tokens across 14 tracked hosts, and a dedicated H100 rents for $1.99–$6.16 per GPU-hour on the published rate cards of Together, DeepInfra and CoreWeave. All figures below are from the vendors' own pricing pages, checked 2026-11-15, cross-checked against independent price trackers.
Buy tokens by default; rent GPUs only past the break-even. The break-even is arithmetic, not opinion: a dedicated H100 at DeepInfra's own $2.90/GPU-hour beats its own $0.89-per-1M serverless output price only if you push more than 3.26 million output tokens per hour, sustained (about 904 output tokens/second). For a model it serves at $0.32-per-1M output, the bar rises to 9.06M tokens/hour. Spiky or low-volume traffic: serverless wins before you do any math. Steady, saturating, multi-tenant load: hourly starts winning — and the commitment ladder ($1.99 → $3.99 → $5.49 for the same H100 at Together) prices exactly who eats the idle risk.
| Inference provideryou buy tokens | GPU cloudyou rent the hardware | |
|---|---|---|
| Pricing | Per 1M tokens. DeepSeek-V3 output: DeepSeek's own API $1.10, DeepInfra $0.89, Fireworks $0.90, Together $1.25, Hyperbolic $0.20 — 14 tracked hosts, input $0.20–$1.45, output $0.20–$1.50. $0 between requests | Per GPU-hour. Dedicated H100: DeepInfra $2.90, Together GPU clusters $1.99 preemptible / $3.99 on-demand (Dedicated Inference tier $5.49), CoreWeave 8-GPU node $49.24 = $6.16 per GPU. Idle time bills at full rate |
| Model | Whatever the host serves — open-weights families (DeepSeek, Llama, Qwen, GLM, Kimi) dominate. You accept the host's quantization, context limits and scheduling | Whatever you deploy — open weights by necessity, since you load them. You pin the version, choose the quantization, and own the serving stack |
| Context window | Whatever the host lists per model (DeepSeek-V3 at DeepInfra: 160K). Long context is a token bill, not a capacity you own | Bounded by the VRAM you rent. You can extend context, cache aggressively, and keep sessions warm at your own engineering cost |
| Chinese support | Model choice decides — DeepSeek and Qwen hosts are strong Chinese performers; the provider itself is language-neutral | You deploy the model, so language support is exactly the checkpoint you chose |
| API | OpenAI-compatible chat completions nearly everywhere; switching hosts is a base-URL change. DeepInfra adds per-request service tiers (Standard 1×, Priority 1.5×, Flex 0.8×) | Not included. You bring or adopt a serving stack (vLLM, SGLang, TensorRT-LLM); the hourly rate buys silicon, not an endpoint |
| Pros |
|
|
| Cons |
|
|
| Score | 8.8 | 7.6 |
What the landing pages won't tell you
The provider market is itself a market, and most buyers never shop it. One model — DeepSeek-V3 — has 14 tracked hosts, and the output-price spread runs from $0.20 (Hyperbolic) to $1.50 (Nebius) per 1M tokens: a 7.5× gap for identical weights. The same model costs $1.10 output on DeepSeek's own API and $0.89 on DeepInfra. Price the host, not the model.
One GPU can carry three prices in one rate card. Together lists the same HGX H100 at $1.99 preemptible, $3.99 on-demand and $5.49 in its Dedicated Inference tier. The ladder is not about the silicon — it prices who eats the idle risk: preemptible means you accept eviction, dedicated means the host guarantees capacity and you fund the guarantee.
Utilization decides, and the formula is one division (ours): hourly rate ÷ price per 1M output tokens = output tokens per hour you must sustain. At DeepInfra's own rates, $2.90 ÷ $0.89 = 3.26M tokens/hour (~904 output tokens/second) before a dedicated H100 beats its own serverless price. Against its $0.32 Llama-3.3-70B-Turbo output rate, the bar rises to 9.06M/hour (~2,517 tok/s). Run the division before you rent anything.
The verdict
In November 2026 the default is to buy tokens: the serverless market for open-weights models is liquid, priced to the hundredth of a dollar, and charges you nothing between requests. Rent a GPU only when your load is steady enough to survive the division: hourly rate ÷ output price per 1M tokens = the sustained tokens-per-hour bar. At DeepInfra's own published rates that bar is 3.26M output tokens per hour on DeepSeek-V3 and 9.06M on Llama-3.3-70B. If your forecast clears it with headroom, take the hourly route — and prefer the commitment tiers only when the workload is genuinely 24/7, because the $1.99–$5.49 ladder for the same H100 is the market charging you for certainty. Note what one card can hold: DeepSeek-V3's 671B parameters exceed a single H100's 80 GB even at 8-bit, so "dedicated DeepSeek-V3" means multi-GPU serving, not one card.
Straight answers
What is the difference between an inference provider and a GPU cloud?
An inference provider bills per token and serves a model it operates — you send an OpenAI-compatible request, it returns text, and between requests you pay nothing. A GPU cloud bills per GPU-hour and gives you the raw accelerators — you install the weights, run the serving stack, and the meter runs whether the card is busy or idle. On November 2026 rate cards, DeepSeek-V3 output costs $0.20–$1.50 per 1M tokens across 14 tracked hosts, while a dedicated H100 rents for $1.99–$6.16 per GPU-hour on Together, DeepInfra and CoreWeave's own price pages.
When is renting a GPU cheaper than buying tokens?
When your sustained output volume clears the break-even: hourly rate ÷ price per 1M output tokens = output tokens per hour. Using DeepInfra's own published numbers, its $2.90 H100-hour must carry more than 3.26M output tokens per hour (~904 tok/s, sustained) to beat its own $0.89-per-1M DeepSeek-V3 output price; against its $0.32 Llama-3.3-70B-Turbo rate the bar is 9.06M tokens/hour. Monthly, a $2.90 H100 costs $2,088 for 720 hours — the token-equivalent is about 2.35B output tokens per month at DeepSeek-V3 rates.
Can one H100 actually serve DeepSeek-V3?
No — and this matters for reading the break-even. DeepSeek-V3 has 671B parameters; at 8-bit precision the weights alone are roughly 671 GB, far beyond one H100's 80 GB of HBM. A dedicated DeepSeek-V3 deployment means multi-GPU serving, so the per-GPU break-even for that model is a sizing thought experiment. For a model that fits one card — Llama-3.3-70B at 8-bit is roughly 70 GB — the arithmetic is directly actionable, which is why the 70B break-even (9.06M tokens/hour at DeepInfra's rates) is the honest planning number.