Why an AI agent costs 30 times a chatbot

Because an agent rereads its whole context at every step. A model has no memory between two calls: whatever was said three steps earlier has to be sent again in full at the fourth, and paid for again at full price. A five-step task costs roughly three times a chatbot answer, and the ratio passes thirty beyond fifty steps.

This is the mechanism behind most of the budget surprises of 2026, including at teams that did nothing unreasonable. One developer ran up 4,200 dollars of API calls over a long weekend on a single autonomous session, and the median monthly spend of an equipped developer sits at 480 dollars, against 80 at the tenth percentile.

Where the money goes, line by line

The breakdown is remarkably stable from one system to another, and it surprises almost everyone the first time. Context resent at every step is 62 % of the bill, tool definitions 14 %, the reasoning actually produced 11 %, the system prompt 8 %, and retries that served no purpose 5 %.

The part you think you are paying for, in other words, the part where the model thinks and writes, is one ninth of the total. The remaining eight ninths are repetition: the same instructions, the same tool descriptions and the same history, reread in full on every turn of the loop.

The 14 % spent on tool definitions deserves a second look, because it is the line item nobody thinks to question. Every tool an agent could theoretically call is described in full on every step, whether it is used or not, so a product that has connected thirty integrations pays for all thirty on a task that touches one. This is the strongest practical argument for narrow, well-scoped agents, and it is the same reasoning that makes an auto-generated skill worse than no skill.

That imbalance has a direct consequence for how you optimise. Shortening answers achieves almost nothing, while reducing what enters the context at every step acts on the main line item. It is also why memory is not only a quality question: a memory that swells costs money on top of degrading answers.

Caching changes everything, and it is more fragile than it looks

Providers charge around 90 % less for the part of the context they recognise as already processed. On a three-thousand-unit system prompt repeated fifty times, the bill falls from 0.45 to 0.053 dollars, 88 % less for a change that alters no answer at all.

The constraint is brutal: the match has to be exact, byte for byte, across the whole beginning of the context. A single differing character anywhere in the prefix breaks the hash and puts you back on full price, with no error message to warn you that you have just lost the cache.

What breaks it is nearly always unintentional. A timestamp placed at the top of the prompt, tool definitions serialised in a different order from one call to the next, trailing whitespace, a model switched mid-session: each of these details, individually harmless, multiplies the bill tenfold on the most repeated part of the system.

What we had to change in our own agent

We found the subject in the most ordinary way possible, looking at a bill that would not go down although we had shortened the answers. Our prompt prefix carried the current date, which seemed useful so an agent would know when it was working, and it was enough to invalidate the cache on every call.

The rule we have applied since lives in the code rather than in a document, because a rule of that kind is lost within three weeks if it depends on somebody being vigilant. Everything that varies, the date, the conversation identifier, the state of the open file, goes after the stable part, and the serialisation order of tools is frozen instead of being left to the data structure that carries them.

The second change was to treat the silent retry as a design fault rather than merely a cost. The 5 % of the bill that goes into useless attempts is above all the symptom of an agent starting over without saying so, which is exactly the behaviour a good agent must be incapable of. An agent that fails openly costs less than an agent that insists.

The paradox: prices collapse and bills rise

The cost of the same level of answer has fallen by a factor of 280 in eighteen months, and it is a real, measurable fall that has come with no fall in enterprise bills at all. The phenomenon is old, it has a name in economics, and it holds here without nuance: when a resource gets cheap, people use more of it.

In practice, a team that puts caching in place and divides its cost per step does not hand the savings back. It lengthens loops, grows contexts, runs experiments it used to forbid itself, and finds itself six weeks later with the same bill for a system doing considerably more. That is a good outcome, provided nobody promised a reduction.

There is a consequence here for buyers that is worth stating plainly. A vendor offering to pass consumed units through to you is transferring a risk you can neither forecast nor steer, and one that depends entirely on engineering choices you cannot see. The price should sit on the work delivered, for the same reason it should not sit on a number of seats.

What to ask before signing

Three questions are enough to tell whether a vendor has worked the subject or wired up a model and looked away.

What does a typical task cost, at your end, today? A vendor who has instrumented their system answers with a figure and a spread; a vendor who has not looked answers by talking about the price of models, which is not the question asked.

What happens when a task goes into a loop? The right answer describes a hard limit, a stop and a readable trace. No answer means there is somewhere a case in which your agent runs all night, and you will learn about it on the invoice.

How does cost evolve as a conversation lengthens? A product with no provision watches its spend grow with the square of the length, because every step resends everything before it. A product that has worked the subject summarises, prunes and expires, and can tell you at what threshold it does so.

What this says about the vendor’s trade

The economics of consumed units are becoming a competitive advantage again, after three years in which they mattered to nobody because everyone was paying negligible amounts. Two products wired to the same model, delivering the same service, can differ by a factor of ten on cost of goods, and that gap always ends up showing in the price or in the margin.

It is also what makes comparing agents harder than comparing conventional software. An agent’s cost of goods depends on internal decisions invisible from outside, exactly like its forgetting policy, and a demo shows none of it. The logical next step is therefore the trial: how to test an AI agent before signing, measuring cost per task as much as answer quality.