How AI Agents Actually Work

An agent is a loop. The model receives a goal and the current context, proposes an action, a runtime executes that action, and the result is appended to context. The loop repeats until the model stops or a limit is hit. The model itself does nothing between turns.

The loop, step by step

Start from what actually executes, because the word agent hides it.

A runtime holds a conversation: a system prompt, the user goal, and a growing list of turns. It sends that to a model along with a set of tool definitions, each of which is a name, a description, and a parameter schema.

The model returns one of two things. Either text, meaning it is done or asking a question, or a request to call a tool with specific arguments.

If it is a tool call, the runtime executes it. The model does not run anything. It emits a structured request and your code decides whether and how to carry it out. The result, or the error, is appended to the conversation as a new turn.

Then the loop runs again with the longer conversation. And again. Until the model returns plain text, or a turn limit is reached, or a budget is exhausted, or a human interrupts.

That is the whole mechanism. There is no persistent process thinking between turns, no memory that is not text in the conversation, and no ability to act that the runtime did not grant.

Once that is clear, most agent behaviour stops being mysterious. An agent that forgot something has a context problem. An agent that did something dangerous had a tool that permitted it. An agent that looped forever had no adequate stop condition.

The four parts you actually build

The tool set. Each tool is a description the model reads and a function your code runs. The description is doing more work than people expect: it is the only thing telling the model when this tool applies. Vague descriptions produce tools that are never selected or selected wrongly, and a tool that is never selected is dead weight in every request.

The context policy. What goes into the conversation and what gets removed. Every turn accumulates, so a long run either overflows the window or costs a fortune. The policy decides what is summarized, what is dropped, and what is fetched on demand rather than carried.

The stop conditions. A maximum number of turns, a spend ceiling, a wall clock limit, and a definition of done. Without these an agent that misunderstands the goal will keep working, and it will keep billing.

The permission boundary. What the tools can reach: which files, which network destinations, which credentials. This is the only real safety control, because instructions can be argued past and permissions cannot.

Notice that three of the four are about constraining the loop rather than about the model. That ratio is roughly right for how the work actually divides.

Why multi-agent setups are just nested loops

A planner delegating to workers is one loop whose tool happens to be another loop. This is worth knowing because it means the same four concerns apply at every level, and the costs multiply rather than add. Two agents are not twice the cost of one, because the outer loop pays for the inner loop's context as well as its own.

Where the cost and the errors come from

Both come from the same place: context.

Every turn resends the whole conversation. A ten turn run does not cost ten times the first turn, it costs roughly the sum of a growing window, which is closer to quadratic than linear. That is why agents feel cheap in testing and expensive in production, where runs are longer.

Accuracy degrades from the same cause. A context holding fifty thousand tokens of file dumps, tool output, and earlier reasoning contains a great deal that is irrelevant to the current step, and irrelevant material competes for attention with the material that matters.

So the single highest leverage decision in agent design is what never enters context at all. An agent that retrieves the specific passage it needs is cheaper and more accurate than one that reads a directory and hopes. This is the reason a retrieval layer sits under a well built agent rather than beside it: RDK's stacked retrieval resolves most queries before the model is involved, so the loop carries what it needs and nothing else.

If you build one thing carefully, build the context policy. The model is a component you rent. The loop is the part you own.

Frequently asked questions

How does an AI agent work?
As a loop. The runtime sends a goal, the conversation so far, and a set of tool definitions to a model. The model returns either text or a request to call a tool. The runtime executes the tool and appends the result to the conversation, then repeats until the model stops or a limit is reached.
Does the model execute the tools itself?
No. The model emits a structured request naming a tool and its arguments. Your runtime decides whether to carry it out and does the work. That separation is why permissions, not instructions, are the real safety boundary: the model can ask for anything and only the runtime can grant it.
Why do agents get expensive on long runs?
Because every turn resends the entire conversation, so cost grows closer to quadratically than linearly as turns accumulate. Agents therefore look cheap in short tests and expensive in production, where runs are longer and each step carries everything that came before it.
What causes an agent to loop forever?
An inadequate stop condition. The loop ends when the model returns plain text, or a turn cap, spend ceiling, or time limit fires. Without those, an agent that has misunderstood the goal will keep proposing actions indefinitely, and it will keep billing for each turn.