The Bill Nobody Adds Up

Two reading levels: the running text is the thread — for everyone, no prior knowledge required. The specially marked deep dive boxes are elaborations for specialists and can be skipped without losing the thread.

Someone buys a graphics card. Two thousand, three thousand euros, a fan the size of a brick. The thought behind it is always the same and sounds reasonable: Once the thing is sitting here, every query costs me nothing more. No subscription, no token bill, no American corporation holding out its hand at the end of the month. It feels good. It is also, in most cases, a miscalculation — and not narrowly so, but by a wide margin.

By the end of this series you will know whether your case really needs to be local — and if not, where to go instead. No hype, no panic, with numbers a pocket calculator will not overturn.

To get there I take apart the four reasons usually given for “local” — price, performance, licence, provenance — one per part. What remains is a circle of cases in which local really is the only option — and it is far smaller than the debate makes it seem. It can be marked out in advance; whoever does not mark it out buys hardware for a problem they do not have. Let us begin with price, because that is where almost everyone begins.

The price everyone compares — and why it lies

Open any model price list and you see two numbers: so much per million input tokens, so much per million output tokens. It looks like a price tag in the supermarket, and that is exactly how it is used: model A costs three dollars, model B costs ten, so A is cheaper. Done.

Only a token is not a kilogram. Two models can burn wildly different amounts of them for the same task — and then the unit price says nothing anymore. A current example: the Chinese model Kimi K3 costs roughly half as much per token as OpenAI’s GPT-5.6 Sol — three dollars instead of five per million input tokens, fifteen instead of thirty on output. On paper a clear price advantage. But run both through the same benchmark, and that advantage shrinks from half to a few per cent: about 2,700 against 2,800 dollars for the complete run. The reason is in the measurements of the analysis firm Artificial Analysis — Kimi is verbose. It burns 130 million tokens where GPT gets by on 70 million and the average across all models sits at 63 million. Even Claude Fable 5, no model of few words itself, stays below that at 87 million — Kimi’s low unit price is all but swallowed by its high consumption.

That is not a quibble, it is the whole point. The token price is the number in the marketing. The number on your bill is a different one.

What the task really costs

The more honest measure, then, is not “euros per token” but “euros per task”. Artificial Analysis works this out: an averagely expensive top model costs, averaged across a whole benchmark run, on the order of a good two dollars per task — not because the token is expensive, but because modern models think, and thinking means burning tokens. A model that broods for a long time before it answers is, at identical unit price, the more expensive one.

For you as a user that turns the question around. Not “Which model has the lowest token price?”, but “Which model handles my kind of task with the fewest tokens at sufficient quality?”. That is a question you have to measure, not skim. And it is the first place where the naive price calculation tips over: a verbose model saves on unit price and pays it back on volume — no matter where it runs.

The only number that counts: what stands there at the end

There is a third level, and it is the only one that truly counts: what did the finished thing cost? Not the token, not the single task — the complete project, from the first line to the running result.

A developer recently worked this through in public by having a whole app built by an AI model and laying the bill bare: a good ten million input tokens, but only around 70,000 output tokens — and at the bottom line about ten dollars. The reason for the low figure is inconspicuous and decisive: in this case around four out of five requests were repeats, which the system answered from a cache instead of recomputing them. A single caching effect, which no price list shows, pushes the real bill down by a multiple.

And here it pays to look at something the first two cost levels left open: how many tokens a model burns and how often the cache catches decide the choice of model — not the location. Both travel with the model, whether it runs in the cloud or on your card. Exactly one quantity separates local from cloud, and it is on no price list: utilisation. Let us work it out.

Our own bill, laid bare

Take a realistic scenario: a machine with a decent graphics card for around 5,000 euros, written off over three years, running a good open model on it (Qwen3-30B), 600 watts under load, German commercial electricity at 27.15 cents per kilowatt-hour. No cheap industrial tariff. So what does the million tokens cost then?

Utilisation of the card	Cost per million tokens
24/7 at full load	0.51 €
8 hours × 250 working days	1.44 €
10 % (continuous service for a small team, ~2 hrs/day of load)	2.99 €
1 % (realistic office day)	27.70 €

The spread is the whole point. A card you have bought costs the same whether it computes or stands idle — the purchase is paid, the electricity runs, the depreciation ticks. Its price per token therefore depends almost entirely on how well utilised it is. A data centre runs its cards around the clock for hundreds of customers at once. Your office machine stands idle most of the time — because between requests there is reading, typing and thinking, and the card, on a daily average, rarely gets above one per cent utilisation. At that one per cent you do not pay cents but almost thirty euros per million tokens.

Now the fair objection a developer raises at this point: I already have the workstation, the graphics card is all that costs me extra. True — count only a GPU at around 2,500 euros instead of the whole machine, and the best case sinks from 0.51 to about 0.37 euros per million tokens. And even then your own card does not come out ahead of the cheapest cloud model (DeepSeek V4 Flash, 0.26 euros, at better quality); it sits slightly above it. The one comparison it clearly wins is against an equivalent, Europe-managed endpoint. That costs, for the same model, around 2.70 euros — your own card at full load about a fifth of that.

With that, the only case in which price really argues for local is cleanly outlined: a card you already own, that is utilised around the clock, compared against the European endpoint rather than the cheapest provider in the world. Remove just one of those three conditions — and at an ordinary desk all three are missing — and the calculation tips. Whoever justifies local on price must therefore say exactly whom they are counting against and how full their card runs. Otherwise they lose the argument to a pocket calculator.

Deep dive · for the technically inclined The 0.51 euros are a best case with two hidden assumptions. First, they rest on the single-stream throughput (192 tokens/s) — a data centre bundles many requests (batching) and gets a multiple out of the same card, which drives its price down further. Second, “full load around the clock” is a fiction for a single workstation. The maxim that carries the whole economics: a model’s active parameters are the labour cost per job, its total parameters are the rent on the warehouse. A “mixture of experts” like Qwen3-30B activates only a fraction of its parameters per request (cheap job), but still has to fit entirely into the expensive memory (full rent). This double edge is the core of every local cost question — and the reason Part 2 looks at what even fits on your machine in the first place.

Start with the need, not the model

If price is not the argument — where is the real saving? Answer: one step earlier, at the question hardly anyone asks before they pick a model. Namely: What task do I actually have, and how big does the tool for it really need to be?

Because the most expensive habit is to reach for the biggest model for everything. Very many tasks in an AI system are not thinking tasks at all but sorting tasks: classify text, route it, summarise it. A tiny model, post-trained on exactly this one task, reaches hit rates that a seventy-times-larger general-purpose model does not get near without training — at half a GPU-hour of training effort. The graphics-card maker NVIDIA estimates that in typical AI agents 40 to 70 per cent of all model calls can be replaced by such small specialists. And the most solidly documented saving is plainer still: do not recompute repeated or similar requests at all, but serve them from a cache. In a cleanly measured series that lowers costs by 86 per cent — at over 90 per cent hit rate.

That is the unromantic truth behind the cost question: the big savings do not come from where the model runs, but from taking the right-sized model and sparing yourself unnecessary calls. Both work in the cloud just as they do on your own card.

Deep dive · for the technically inclined Beware a popular number: “model routing saves 85 per cent” circulates readily but is the best case of a single benchmark. In robust measurements automatic routing really brings a factor of 1.4 to 3.7 — decent, but not an order of magnitude. The 86 per cent above, by contrast, does not come from routing but from the semantic cache, measured over nearly 64,000 requests. Anyone planning savings should not confuse the two — the cache is the more reliable lever.

The uncomfortable objection

Honesty is part of it, even when it runs against one’s own sympathies. For there is a finding that contradicts the self-built, self-hosted approach as a whole — and it is no side detail. In practice, bought-in AI solutions go into production noticeably more often than self-built ones; in a broad survey roughly two-thirds of the purchased projects made it across the finish line, but only about a third as many of the self-developed ones. That is not a statement about technology, but about effort, maintenance, and how often an ambitious in-house venture stalls somewhere between prototype and daily use.

For the cost question that means: alongside the 0.51 or 27.70 euros per million tokens comes a second item that appears on no electricity bill — the person who runs all of this. Part 2 works that item out. This much in advance: it is more expensive than the graphics card.

What this means for us

Price is thereby finished as an argument for “local” — not because local AI is expensive, but because it does not make things cheaper than the obvious alternative. Three thoughts to take away:

Never compare the token price alone. It is the marketing number. What counts is consumption per task and the bill for the finished project — and that can come out higher at a low unit price, lower at a high one.
Utilisation beats acquisition. Your own card pays off on price only if it belongs to you anyway and almost never stands idle. At a typical workplace it almost always stands idle — and thereby becomes the most expensive route there is.
The real saving lies before the choice of model. The right-sized model and a cache for repeats bring more than any location decision — and both are available in the cloud just as at home.

If price is not the reason, then why go local at all? Perhaps for the performance — because you want the best and believe open models are by now on a par. Whether the card under your desk really keeps up, and how large the gap to the top really is, we take on in Part 2.

If this piece gave you something to think about, feel free to share it — and at the next offer that advertises with “runs entirely on your side, saves the cloud costs”, ask yourself which of the three numbers you are actually being shown: the token price, the cost per task, or what really stands on the bill in the end. This series has four more parts — performance, licence, provenance, and what remains at the end. If you do not want to miss any of them, subscribe to the newsletter: a short note the moment a new piece appears, no promotional newsletter, unsubscribe at any time with one click.

Sources (selection):

Artificial Analysis — model pages for Kimi K3, GPT-5.6 Sol and Claude Fable 5 (Intelligence Index v4.1: token consumption Kimi 130M / GPT-5.6 Sol 70M / Fable 87M / average 63M; total cost of the run Kimi $2,709.75 vs. GPT-5.6 Sol $2,824.18; prices Kimi $3/$15, GPT-5.6 Sol $5/$30 per million tokens): https://artificialanalysis.ai/models/kimi-k3 · https://artificialanalysis.ai/models/gpt-5-6-sol · https://artificialanalysis.ai/models/claude-fable-5
Own local cost calculation (Qwen3-30B-A3B, 192.4 tokens/s measured, 5,000 € / 3 years, 600 W, commercial electricity 27.15 ct/kWh per wattline/BDEW): 0.51 € (24/7) to 27.70 € (1 % utilisation) per million tokens.
DeepSeek V4 Flash — API price around 0.26 € per million tokens (output price, matching the output throughput of the local calculation), higher quality index than the locally run model.
OVHcloud AI Endpoints — managed EU hosting, Qwen3.6-27B around 2.70 € per million tokens: https://ovhcloud.com/en/public-cloud/ai-endpoints/
NVIDIA — “Small Language Models are the Future of Agentic AI” (40–70 % of agent model calls replaceable by small specialist models).
AWS — “Semantic caching: Impact and benchmarks” (−86.3 % cost at 90.3 % hit rate and 91.2 % accuracy, 63,796 requests): https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/semantic-caching-benchmarks.html
arXiv 2505.16078 — post-trained Llama-3.2-1B beats 70B zero-shot at classification (0.999 / 0.892 / 0.865) at 0.5 GPU-hours of training.
S&P Global / 451 Research (via genai-roi dossier) — bought-in AI solutions go into production roughly three times as often as self-built ones.