The number everyone quotes about AI costs is that token prices are collapsing. A 2026 analysis of the token market tracked a roughly 600-fold decline from GPT-3’s launch price of $60 per million input tokens in June 2020 to under $0.10 per million by early 2026. Economy-tier prices now halve roughly every 1.1 years, faster than Moore’s law. The story is true, and it is also the most misleading number in the industry.
What the deflation headlines hide is the part that actually sets your bill: how many of your GPU hours are doing useful work. Inference is billed by the token, but served by the hour.
The number teams miss
Run a model on a rented GPU and your cost per token depends mostly on how full that GPU is. A measurement study published this year put effective cost on identical H100 hardware anywhere from $0.21 to $15.25 per million output tokens, driven by a single variable: the request rate hitting the box.
| Same H100, same Mixtral 8x7B model | Effective cost per 1M output tokens |
|---|---|
| 1 request per second (near idle) | about $15.25 |
| 25 requests per second | about $0.87 |
That is a 17.5x spread on identical hardware, and the near-idle penalty reaches 36x across other configurations. The gap between those two rows is wider than the difference between most models you were comparing, and it never appears on a pricing page.
The reason is straightforward. A rented GPU is billed by the hour, not by the token. If it sits at 40 percent utilization, your effective cost per token roughly doubles versus the same box running full. The sticker price of the hardware did not move. The idle time did.
This is why two teams running the same model can quote wildly different unit costs. One runs a bursty chatbot at 40 percent utilization; the other runs a saturated batch pipeline at 90 percent. The model, the hardware, and the sticker price are identical. The unit cost is not. When a vendor quotes you a cost per token, ask what utilization assumption is baked into it.
Open-weight models made this worse
The open-weight wave added a specific twist. Mixture-of-experts models activate a small slice of their parameters on each token, but you have to hold all of them in memory. Run a 400B-class MoE model yourself and you pay for VRAM across every parameter while a fraction does the work on any given token. A managed API spreads that memory cost across thousands of customers and thousands of concurrent requests. You cannot.
This is why the self-hosting crossover is not a clean line. A 2026 cost analysis put the break-even at roughly 2 to 5 million tokens per day on reserved capacity, and only if you keep the GPU busy. Below that, the API wins because idle time and engineering overhead eat the savings. Above it, on a predictable workload, self-hosting beats even the cheap open-weight hosted APIs.
The second hidden cost: time of day
There is a second line item that gets almost no attention in India: time-of-day pricing. Some providers bill peak and off-peak rates. DeepSeek’s own API charges a peak rate about 2x its off-peak price during Indian working hours (06:30-09:30 and 11:30-15:30 IST). The same request costs twice as much depending on when your users happen to be awake.
For a team building for an Indian audience, peak traffic lands exactly in their peak rate. You do not get to choose when your customers call your API. If your traffic spikes at 10am and 3pm, you are paying the peak rate on exactly the requests that matter most to your users.
What actually moves the number
If you are sizing an inference budget, work down this list before you compare model prices:
- Utilization before hardware. A 40 percent utilized H200 can cost more per token than a fully loaded H100.
- Cache aggressively. Cache-hit input tokens bill at a fraction of cache-miss rates. A long system prompt or shared context is free money if you cache it.
- Read the peak rate, not the sticker rate. Time-of-day pricing can double a workload’s cost.
- Batch what you can. Continuous batching is nearly free for non-interactive jobs like RAG indexing or batch extraction.
- Judge MoE models by active parameters, not total. The self-hosting math changes completely.
How IndieRouter fits
The managed inference argument is simple once you accept that utilization is the real cost. IndieRouter runs shared Indian GPU capacity and bills a flat per-token rate around the clock: no peak surcharge, cache-hit input at ₹1 per million tokens on the flagship DeepSeek V4 Flash, and INR billing with GST invoices. The idle time, the memory overhead, and the utilization problem sit on our side of the fence. You pay for tokens, not for the GPU sitting dark at 3am.
We say the caveat out loud. If you run millions of tokens a day on a reserved, fully loaded cluster, buy your own hardware. Managed inference is the right default for everything below that crossover, and for any team that does not want to own the ops, the queues, and the empty hours.