There is a whole category of projects companies shelved between 2023 and 2025 with one sentence: “the numbers don’t work”. That folder is worth reopening, because the numbers have moved by more than an order of magnitude.
The fall, measured
GPT-4 class performance, which sold for roughly $30 per million tokens in early 2023, goes for under a dollar today. Silicon Data’s LLM token expenditure index recorded an all-time low of $0.97 per million tokens in early September 2026, more than half below its summer peak: average inference cost fell from about $2.04 at the end of May 2026 to the $1.16–1.18 range by early August, and kept going.
Few commodities in the history of computing have fallen at that rate.
Why it matters more than it looks
A 30x price change is not a margin improvement. It is a change in what is possible at all.
At $30 per million tokens, it only made sense to use the model at the point of maximum value: a one-off query, a final draft, an expensive decision. Everything else ran at a loss.
Under a dollar, the design changes:
- You can afford to process the whole volume, not a sample. Classifying last year’s 400,000 tickets stops being a project and becomes an afternoon.
- You can afford to let the system be wrong and retry. Token-hungry patterns — self-consistency, cross-verification, a second model reviewing the first — stop being an academic luxury.
- You can afford expensive preprocessing. Enriching documents at ingest, generating summaries at several levels, maintaining redundant indexes.
The architecture that was wasteful in 2024 is the correct one now.
The trap: volume eats the saving
This is where companies come unstuck. Unit price falls, yes — but agentic patterns multiply consumption per task. An agent that reasons, calls tools, reads results and retries can burn fifty times the tokens of a single call solving the same thing.
The result is a bill that climbs while the per-token price drops. That is not a contradiction: it is Jevons’ paradox applied to compute.
What we do on projects: measure cost per business outcome, not per token. “What it costs us to process an invoice end to end” is a useful metric. “What a million tokens costs” tells you nothing about whether the project holds up.
And the other exit: making it free
For high volumes and narrow tasks, on-device or self-hosted inference costs zero per query after the initial model download. It is not the answer to everything — small models do not reason like frontier ones — but for classification, extraction, routing and structured drafting, the quality gap no longer justifies paying per call.
It also settles privacy conversations that used to block entire projects: if the data never leaves your network, half a dozen legal objections disappear.
What to do this week
- Retrieve the use cases you killed on cost. Redo the maths at today’s prices. Some of them will clear.
- Instrument cost per business transaction, not per API call.
- Put a ceiling and an alert on every agentic flow before it reaches production, not after the first invoice.
- Evaluate a small self-hosted model for the high-volume, low-reasoning tasks. They are usually 80% of the traffic.
Cost has stopped being the reason an AI project doesn’t happen. When someone tells you it doesn’t pencil out, ask to see the numbers — they are usually two years old.