The problem
Your AI automation cost $340 last month. This month it is $2,900. Nothing obvious changed, nobody launched anything, and now finance wants an explanation you do not have.
I see this regularly, usually right after a system starts getting real usage. And the reaction is almost always to panic and consider ripping the whole thing out.
After 4 years running LLM systems in production, the bill is nearly always fixable without touching what the system does. Here is where the money actually goes.
The scale of the waste
Enterprise spending on LLM APIs hit an estimated $8.4 billion in 2025. Audits find that 40 to 60% of that token spend is waste.
Not "could be optimized." Waste. Tokens that bought nothing.
That number tracks with what I find when I audit a client's AI spend. The first pass usually cuts half the bill with zero change in output quality.
Why it spikes without warning
Token costs are variable in a way most software costs are not. They respond to user behaviour, prompt changes and feature additions, none of which show up as a line item anyone reviews.
A feature launch can triple the bill inside a week. An internal tool getting adopted by a second department does the same. Automation hitting unanticipated volume does it too.
The cost is invisible until the invoice arrives, which is why it is nearly always a surprise rather than a trend someone watched.
The six places it leaks
1. Model selection
Frontier models can cost over 100 times per token what a budget model costs.
Most systems I audit run every single call through the most capable model available, including the calls that classify something into one of four buckets. That classification does not need a frontier model. It needs a cheap, fast one, and the accuracy difference on a constrained task is often nil.
Route by task difficulty. The hard reasoning step gets the expensive model. Everything else does not.
2. Context bloat
This is the biggest one, and it is why context engineering is a cost discipline as much as a quality one.
Every irrelevant token in the window is billed. Teams stuff full CRM records, entire documents and complete conversation histories into calls that needed a few labelled fields.
You pay for that twice: once on the invoice, and again in degraded accuracy, because model performance measurably drops as the window fills with noise.
Trimming context is the rare optimization that cuts cost and improves output at the same time.
3. No caching
A surprising share of production calls are near duplicates. The same question, the same document, the same enrichment lookup run again because nothing remembers the last answer.
Semantic caching can cut API costs by up to 73% on workloads with repetition. Most automation workloads have far more repetition than their builders assume.
4. Retries and loops
An agent that fails and retries burns tokens on every attempt. If a step silently fails 30% of the time and retries three times, you are paying four times for one result.
Failed calls do not show up as failures on the invoice. They show up as volume. Instrument retry rates or you will never see this.
5. Gateway margins
Multi provider gateways typically add 5 to 15% on top of provider rates. Sometimes it is disclosed as a platform fee, sometimes it is folded into the per token price as credit conversion.
Worth knowing what you are paying for the convenience. Sometimes it is clearly worth it. Sometimes you are paying a margin for a routing feature you never turned on.
6. Reprocessing the same input
Systems that re-run enrichment, re-summarize the same document, or re-analyze the same record on every execution, because storing the result was never designed in.
Compute once, store the result, reuse it. Obvious, and routinely skipped.
How I would approach an audit
Start by finding out where the money goes, because almost nobody knows. Log token counts per call type, not just totals.
Then in order:
That sequence usually gets the bill down by half or more, and quality tends to go up rather than down because the context work sharpens the inputs.
The wider point
An AI system with runaway costs is not a reason to abandon AI. It is a sign that nobody has owned the economics of the system, which is the same pattern as nobody owning its reliability.
These systems need someone watching them after launch. The bill is just the most visible symptom when nobody is.
If your AI spend jumped and you cannot explain it, the answer is in the logs, and it is usually cheaper to fix than to live with.