How to Reduce LLM Cost
Measure the input and output split first, then cut input, because it is almost always the larger half and the easiest to reduce without losing quality. After that: stabilise prefixes so caching applies, right-size the model per task, cap the agent loop, batch anything that can wait, and retrieve rather than load.
Measure first, in the right unit
Every intervention below helps one half of the bill and does nothing for the other, so acting before measuring is how teams spend a week optimising the smaller number.
Log input, output and cached tokens on every call, with a tag naming the feature and environment. This is a small change and it is the one that makes everything else possible.
Compute cost per completed unit of work. Not per request. If a user task takes eight model calls, the task is the unit that matters for both pricing and optimisation.
Find the concentration. Spend is almost never spread evenly. One feature, one workflow or one customer is usually most of it, and that is where the entire available reduction lives.
What this normally reveals: input dominates, and the biggest single contributor is context that was assembled generously and used barely. That finding is consistent enough to predict, and it determines the order of everything that follows.
The levers, in order of return
1. Send less input. The largest and safest reduction in most systems. Stop loading whole files when a section will do, stop passing entire conversation histories when a summary carries the necessary state, and stop attaching tool definitions for tools this task cannot use. Every token not sent is a token not billed, not cached, and not competing for attention.
2. Stabilise the prefix so caching applies. Put unchanging instructions at the front and variable content at the end. Cached input is billed at a reduced rate because the work genuinely is not repeated, so this is a discount on real savings rather than a pricing trick. Reordering a prompt is one afternoon and it applies to every request thereafter.
3. Ask for less output. Output is the higher rate. Request structured, bounded responses where a bounded answer is what you need, and stop asking for explanations nobody reads.
4. Right-size the model per task. Classification, extraction, routing and formatting rarely need your most capable model. Reserve it for the steps where its judgement is what you are paying for, and route the rest. This is the lever teams reach for first and it belongs here, because it trades quality for cost in a way the first three do not.
5. Cap the loop. A per-task budget, a turn limit and a wall clock limit inside the agent. This does not reduce ordinary spend at all; it removes the tail that produces alarming invoices.
6. Batch anything that can wait. Asynchronous processing is normally cheaper. Plenty of workloads pay a premium for immediacy nobody asked for.
7. Deduplicate work. Identical or near-identical requests recur far more than teams expect. A result cache keyed on the semantic request, not the exact string, removes them.
Why model shopping disappoints
Moving to a cheaper model reduces the rate on both halves and leaves the volume untouched, so a system sending far more context than it needs simply overpays at a lower price. Worse, the accuracy loss often shows up as retries and longer conversations, which add volume back. Fix volume first, then choose the model on merit.
The structural version
The first six levers are tuning. The seventh is architecture, and it is the one that compounds.
Most input in an agentic system exists because the agent was given material in case it needed it. It reads a directory, loads a file set, or receives a large tool result, and the overwhelming majority of that content is irrelevant to the step being taken.
Retrieval inverts that. Instead of assembling context in advance and hoping it covers the task, the agent asks for what it needs when it needs it, and receives the specific passage rather than the container it lives in.
The effect is threefold and that is why it dominates the list. Cost falls because the tokens are never bought. Accuracy improves because irrelevant material is no longer competing for attention. And latency falls because there is less to read on every turn.
This is what RDK is built to do. Stacked retrieval resolves the large majority of queries before the model is involved at all, so the cheapest possible token, the one that never enters a request, becomes the default rather than the exception.
If you do one thing from this page, instrument the split. If you do two, cut input. Everything else is worth having and neither of those two is optional.
Frequently asked questions
- What is the fastest way to reduce LLM cost?
- Cut input, after measuring the input and output split. Input is usually the larger half and most of it is context assembled generously and used barely, so removing it reduces spend without trading away quality, which the alternatives generally do.
- Should I switch to a cheaper model to save money?
- Not first. A cheaper model reduces the rate and leaves the volume untouched, so a system sending far more context than it needs simply overpays more cheaply. The accuracy loss also tends to reappear as retries and longer conversations, adding volume back.
- How much does prompt caching actually help?
- Enough to be worth an afternoon of reordering. Cached input is billed at a reduced rate because the work genuinely is not repeated, so it is a discount on real savings. Put unchanging instructions at the front and variable content at the end so the prefix stays stable.
- How do you prevent an unexpectedly large bill?
- A per-task cost ceiling inside the agent loop, alongside turn and time limits. Provider spend limits protect the account after the fact. Only a budget the loop itself checks can stop a single runaway task before it finishes spending.