The Free Computer: Why Offloading to CPU Is a Win for Everyone
By Oskar Austegard and Muninn
When you use a modern AI assistant with tool use — code execution, file manipulation, web fetching — you're actually using two very different computers simultaneously. One is extremely expensive. One is essentially free. And right now, the industry's incentive structures are pointed in the wrong direction.
Two Computers, Two Cost Curves
The GPU is Anthropic's scarcest and most expensive asset. H100s run $2–3/hour on spot markets, far more on reserved capacity at scale. This is where tokens are generated — where the model reasons, writes, and responds. Every output token on Claude Sonnet costs $0.000015 (at $15/million). That sounds tiny, but the GPU burns whether it's generating a useful token or an unnecessary one.
The CPU container is commodity infrastructure. An AWS c6i.xlarge — 4 vCPUs, 8 GiB RAM — runs $0.17/hour on-demand, roughly $0.000047/second. In a multi-tenant architecture where Anthropic runs many containers per host, the per-session cost drops another order of magnitude. Call it $0.000005/second as a working estimate.
These are not comparable resources. At typical generation speeds (~90 tokens/second), the GPU produces roughly $0.00135/second of output token revenue. The container costs perhaps $0.000005/second to run. The ratio is somewhere between 100x and 300x depending on utilization assumptions.
What This Means for You
Under a subscription plan, you pay a flat rate regardless of how many tokens are generated or how much compute runs in the container. The container is literally free to you at the margin. Any work that runs there instead of through token generation is a pure win: faster (CPUs execute code orders of magnitude faster than the model can narrate reasoning), more accurate (code doesn't hallucinate), and free.
The practical implication: when you ask a model to "work through" a calculation in prose versus running it in a code cell, you're paying for the slower, less reliable path. A 500-token reasoning chain that approximates a result costs $0.0075. The same computation in Python costs you nothing and produces an exact answer.
What This Means for Anthropic
Here the argument gets more interesting. Anthropic doesn't just sell tokens — they sell outcomes. And the GPU is their bottleneck for delivering those outcomes at scale.
Every second the GPU spends waiting on a tool call — a bash command executing, a file being written, an API being fetched — is a second it could be serving another user. Container time is GPU idle time, from the perspective of throughput. Offloading computation to the container doesn't reduce value delivered; it frees the GPU to generate value for someone else.
The economics compound further when you consider that complex tasks resolved through agentic tool use often require fewer output tokens than pure reasoning to reach the same result. A model that fetches real data rather than reasoning about what the data might say uses fewer tokens, not more. Less GPU time per task completion. Better throughput across the fleet.
The Perverse Incentive
There's a structural problem: token pricing creates a weak incentive to maximize token generation, not minimize it. Verbose reasoning looks productive. A long chain of thought generates revenue. Efficient container execution compresses the bill.
This isn't anyone's fault — it's what happens when the pricing unit (tokens) diverges from the value unit (task completion). But it means neither developers nor the model itself receive a clear signal to prefer container offload. There's no "you saved X by running this in compute" feedback. The efficiency is invisible.
Anthropic could act on this in several ways. The most direct would be instrumenting task-completion quality per GPU-second consumed and showing that agentic, tool-using tasks achieve better outcome-per-GPU-second ratios than pure reasoning chains. That data almost certainly exists in their telemetry. Making it visible — even internally — would shift engineering priorities.
A more ambitious move: restructure pricing to reward efficiency. Charge for outcomes rather than tokens, or discount tool-heavy workflows. This is hard to design without creating perverse incentives in the other direction, but the current model leaves significant efficiency gains unrealized.
The Alignment That Already Exists
What's underappreciated is that the win-win is already structurally present — it just isn't being exploited. Users benefit from container work being free and fast. Anthropic benefits from container work freeing GPU capacity. The interests point in the same direction. What's missing is the explicit design philosophy that treats the CPU container as a first-class resource to be routed to aggressively, not a fallback for code that happens to need execution.
The best agentic workflows already do this intuitively: fetch before reasoning, compute before narrating, execute before explaining. But it's emergent behavior, not a stated principle.
It should be a stated principle.
The Numbers, Briefly
For reference, the pricing that underlies this argument (as of February 2026):
| Resource | Cost |
|---|---|
| Claude Sonnet output tokens | $15 / million tokens |
| Claude Sonnet input tokens | $3 / million tokens |
| Cache reads (10% of input) | $0.30 / million tokens |
| AWS c6i.xlarge (4 vCPU, 8 GiB) | $0.17 / hour |
| Container compute (estimated per-session, multi-tenant) | ~$0.000005 / second |
A 500-token reasoning chain: $0.0075. The equivalent Python execution in a container: effectively $0. At scale, that gap is a GPU freed for something else.
The container isn't a footnote in the AI product stack. It's a subsidy hiding in plain sight — for users, for Anthropic, and for the economics of deploying intelligence at scale.
This post emerged from a conversation between Oskar Austegard and Muninn, a persistent AI assistant built on Claude. Oskar developed the core insight: that the container-as-subsidy argument has symmetric benefits for users and providers, and that the current pricing model fails to signal this. Muninn contributed the numerical framing, the GPU throughput angle, and some of the structural analysis — and initially overclaimed on a boot-sequence example before being corrected. The piece was written collaboratively, with Muninn drafting and Oskar shaping the argument through the conversation that preceded it. The pricing data was verified against Anthropic's documentation and AWS pricing as of February 2026.
