Part 11: What the tokens cost, and why users set the price

Once the runs worked, we wanted to know which configuration we would pay to keep running. A tokens-per-second winner did not answer that on its own: an eight-GPU server with one user could leave most of its capacity unused, while a cheap million-token conversation could fail to recall the fact we asked it to retain. We converted the measurements into explicit cost scenarios, then kept latency and correctness beside the price.

One assumption, stated up front

The campaign did not establish a public rental price for this exact node. We therefore treated dollars per GPU-hour as an assumption, not a measured property of the hardware or a Leapswitch quote. The three values below are the campaign’s recorded sensitivity points. Every chart uses $3.50/GPU-hour so differences come from the measured runs rather than a different price assumption for each model.

# scripts/common.py: the only cost inputs
PRICES = [2.25, 3.50, 4.51]                     # $/GPU-hr sensitivity points
usd_per_m_out = gpus * price / (out_tok_s * 3600) * 1e6
usd_per_request = gpus * price * duration_s / completed / 3600      # used for documents and conversations

Short chats: utilization moved the price more than the hourly rate

We charge the full allocated GPU count in these scenarios, including idle replicas. That makes low concurrency expensive. Batching can amortize weight reads and other work across requests, so aggregate throughput can grow faster than the latency seen by each user. The chart shows where that happened in our runs; it does not promise the same scaling for every model or request mix.

What this says, in order of how much money it moves:

1. Batch your users. The cheapest short-chat point we measured anywhere was $0.42 per million output tokens: Qwen3.6-35B-A3B on four GPUs, layout r_par_dep4_a, 256 users (row in final.csv, run 1010-022232). Run as four one-GPU copies, Qwen3.6-35B-A3B costs $17.92 per million output tokens with one user and $0.46 with 256, about 39× cheaper. Median time per output token increased from 4.4 to 11.2 ms. The lower calculated cost came with a latency tradeoff, and it assumes the service can sustain that load.
2. The layout can cost more than the model. At 16 users, Qwen3.8-27B as eight one-GPU copies cost $7.65 per million tokens; the same model split across both islands with TP8 cost $93.19, because TP8 fell off a cliff at exactly that load (182 ms per token, Part 6).
3. An underfilled eight-GPU allocation was expensive in these tests. MiniMax-M3 on all eight GPUs with one user costs about $125 per million output tokens. We have no multi-user short-chat point for it, so we can't say how far that falls.

Long documents: price per document, not per token

Each cell is cents per document · median time to first token. Qwen3.8-27B is dense and ran as four independent one-GPU copies, so a single document uses one GPU while the other three sit idle and are still paid for. That's why its one-at-a-time cost is so high and drops sharply at four. Concurrency helps here too, but much less than for short chats, because a 250K-token prompt is compute-bound prefill: the GPUs are already busy with one of them. Running several at once mostly spreads the same prefill work across more waiting users. For some models, going from 4 to 16 documents at once saved little or nothing per document while the wait for the first token grew several-fold. If you're pricing a long-document service, price the prefill, and cap how many long prompts each server takes at once.

The full-size models (GLM-5.3, DeepSeek-V4-Pro, MiniMax-M3) only have 512K and 1M document points, on 8 GPUs, and those are in Part 5. We don't put them in this chart because the prompt sizes differ.

A conversation grown to one million tokens

For a growing conversation, we changed the accounting unit again. The ramp contains ten turns from 32K to about 1M tokens, with 128 requested output tokens and a hidden-fact check on each turn. Its dollars per completed conversation include that entire sequence. A fresh million-token document is a separate workload and has a separate cost.

• A second user at once barely lowers the price. Two conversations together took 1.8-2.0× as long as one alone on every model, so the cost per conversation fell only 2-10%. Growing a conversation to 1M is dominated by prefill, and prefill already keeps the GPUs busy with one user. The incremental concurrency benefit was much smaller than on the short-chat workload.
• Cheap isn't the same as good. GLM-5.3-Flash is the cheapest conversation here, but it recalled the hidden fact on only 1 of 10 turns in this test, even though it passed every one-shot 1M recall test. Price the model that answers correctly.
• The giants cost dollars, not cents. GLM-5.3 on eight GPUs takes over ten minutes and about $5.27 for one conversation, but recalled the fact on every turn.

The changes that moved a measured cost or a measured wait

The offload row is deliberately labeled differently. Multiplying 114 s and 17 s by $3.50/3,600 gives 11.1¢ and 1.7¢ for one GPU over those durations. Those are wall-time equivalents of the median turn latencies. Eight requests were allowed in flight, but dividing a median latency by eight does not establish achieved cost per request. That requires elapsed time and completed-request counts for the whole workload. The measured result here is the much shorter later-turn wait; we have not promoted it to a precise per-request saving.

What we'd tell someone pricing a service

1. Charge for concurrency you can actually fill. The cheapest token on this node needs dozens to hundreds of users per server. One user on eight GPUs costs tens of dollars per million tokens.
2. Model long-prompt cost per completed request. In these sweeps, extra concurrency often added first-token delay faster than it reduced cost. Set an acceptable wait before selecting the cheapest point.
3. For 1M-token conversations, quote per conversation and check recall. On our runs a full conversation cost from about 30 cents to about five dollars of GPU time at $3.50/GPU-hour, a second simultaneous user hardly changed that, and the cheapest model was not the one that remembered.

The cheapest point was not our final answer

For a service decision, we would first choose a workload and an acceptable wait, reject configurations that fail its correctness checks, and then compare GPU-time cost at a load the service can actually sustain. The campaign’s minimum dollar figure is a measurement under one set of assumptions, not a retail price or a promise of production utilization. Part 12 brings those constraints back together.

Sources

Bracketed ids [S..] refer to the campaign bibliography (docs/SOURCES.md). Numbers not attributed to a source are our own measurements on this node.

1. [S39] GPU Finder, H200 rental prices and live availability
2. [S37] GPUPerHour, NVIDIA H200 rental price tracker (Oct 2026)
3. [S38] Memetik, H200 GPU price per hour by provider (Sep 2026)