How to Run Coding Agents Cheaper Without Making Them Worse
The durable way to cut agent cost is to reduce how often generation happens, not how much each generation costs. Indexing your material so retrieval answers recurring questions drops token spend on repeated work by 80 to 90 percent, while a smaller model or a truncated prompt buys a smaller saving by trading away quality.
Why the usual advice disappoints
The standard cost levers are a cheaper model, a shorter prompt, and more aggressive truncation. Each works a little, and each is a trade. A smaller model is worse at the hard questions. A shorter prompt removes context the agent needed. Truncation is a bet that what you cut did not matter.
They also all leave the underlying pattern intact: the same questions get regenerated indefinitely. If your agents ask about the same conventions every week, you are paying a generation for each asking regardless of how tightly you have tuned the call.
The question worth asking is not how to make each call cheaper but how many calls should be happening at all.
Eliminate the repeats
Agent traffic is dominated by questions with existing answers. What is our error-handling convention, why does this middleware exist, how do we name handlers, what did we decide about retries. These are not novel; they are institutional facts that keep getting regenerated because nothing makes them retrievable.
Index them and they resolve from the private layer without a model call. That layer handles 40 to 65 percent of queries in practice, and the public network adds another 15 to 20 percent, leaving 5 to 10 percent that genuinely needs generation.
That is where the 80 to 90 percent reduction on repeated work comes from, and note what it does not require: no smaller model, no shorter prompt, no quality trade on the questions that still reach the LLM.
Start with what gets asked
The most efficient thing to index first is whatever your team answers most often in chat. Those questions are proven recurring demand, which is exactly the profile where retrieval pays back immediately.
Why it keeps getting cheaper
Prompt tuning is a one-time saving that erodes as your codebase and practices drift. Retrieval compounds in the other direction.
Every question your agents answer well is a candidate chunk. As the index grows, the share of queries resolved before generation rises, so the cost curve bends down over time rather than creeping back up.
The maintenance burden is real but small and mostly editorial: keep the indexed material accurate, because retrieval will return a stale answer with exactly as much confidence as a current one.
Frequently asked questions
- Will this make my agent worse?
- No. It reduces how often the model is called rather than what it is given when it is called. Novel questions still reach the model with full context.
- How much can I expect to save?
- 80 to 90 percent on repeated work. Genuinely novel questions still cost a generation and always will.
- Is this just a bigger context window?
- No. A window pays to carry content on every call. An index pays a lookup only when the content is relevant, which is why it scales past what a prompt can hold.
- What should I index first?
- The questions your team already answers repeatedly. Proven recurring demand is where retrieval pays back fastest.