What Belongs in an Agent Instruction File

What the file is actually for

Coding agents look for a conventions file in the repository and load it into context before doing anything. Depending on the harness it may be read from the repository root, from the working directory, or merged across several levels, with nested files applying to their own subtree.

The defining property is that it is loaded every time. Not when relevant, not when asked for, every run. That makes it the most expensive text in your project per unit of length, and it is the reason length discipline matters more here than anywhere else.

So the question to ask of every line is narrow: does the agent get this wrong without it?

That framing rules out most of what people put in these files. An agent does not need your architecture explained, because it can read the code. It does not need your API documented, because the signatures are right there. It does not need general software advice, because that is in the model already, and restating it competes for attention with the things that are specific to you.

What it does need is the small set of facts that are true of this repository and invisible from inside it. The command that actually runs the tests, as opposed to the one in the README that broke two refactors ago. The convention that errors are returned rather than raised. The directory that is generated and must never be edited by hand. Those are cheap to state and expensive to discover.

What earns its place, and what does not

Commands that work. Build, test, lint, run a single test, start the dev environment. This is the highest value content in the file, because an agent that guesses the command wastes a cycle and sometimes damages state. State the exact invocation.

Conventions the code does not reveal. Where new files go, how errors are handled, which of two competing patterns is current. Codebases usually contain both the old and the new way, and an agent reading the code has no way to tell which one you want more of. This is the single most useful category and the one most often missing.

Hard constraints with consequences. Never edit the generated directory. Migrations are written by hand. Do not add dependencies without asking. These are the rules where being wrong is expensive, and they are worth stating even though the agent might have inferred them.

Where things live, but only where the layout is surprising. Standard layouts need no explanation.

What does not belong: architecture essays, anything the code states plainly, general engineering advice, onboarding narrative written for a human's first week, aspirational rules nobody follows, and long lists of preferences with no stated consequence. Each of these is paid for on every request and returns nothing.

A useful test for any candidate line: has an agent actually got this wrong, in this repository, in a way you remember? If not, leave it out until it does.

Why these files rot, and the loop that prevents it

The decay pattern is consistent enough to predict. The agent does something wrong, someone adds a line forbidding it, and that line stays forever. Repeat for a year and the file is several hundred lines of accumulated corrections, many of which describe problems that no longer exist.

The damage is not only cost. Long instruction sets produce internal contradictions, because rules written months apart by different people were never read together. When an agent is given conflicting guidance it does not report the conflict, it picks one, and which one it picks is not stable. That is the mechanism behind the common complaint that an agent follows the conventions inconsistently.

The maintenance loop that works is unglamorous.

Review on a cadence, not on incident. Once a month, read the whole file top to bottom. Contradictions are visible when the lines are read together and invisible when they are added one at a time.

Delete anything whose failure you cannot recall. If nobody remembers what the rule prevents, it is costing you attention for nothing.

Test by removal. Take out a rule you suspect is obsolete and see whether the behaviour returns. This is the only real evidence, and it takes one session.

Keep it short enough to read in one sitting. A file nobody reads in full is a file nobody can maintain, and length is the thing that makes it unreadable.

The layer this belongs to

The instruction file is one of four ways to give an agent what it needs, and most of the content people put in it belongs somewhere else.

The instruction file is always loaded. Use it for a small set of standing rules and commands.

A skill is a procedure loaded when the task calls for it. Anything that applies to one kind of work rather than all work belongs here, and moving it out of the always-on file removes it from every unrelated request.

An MCP server provides access to something the agent cannot otherwise reach. Access is not instruction, and the two get conflated constantly.

Retrieval supplies knowledge on demand. This is where the bulk of what people write into instruction files actually belongs: how a subsystem works, why a decision was made, what the conventions are across a large codebase. It is knowledge, not instruction, and knowledge does not need to be present on every request to be available on the one request that needs it.

That last distinction is the practical one. Ask whether a line is telling the agent what to do, or telling it something to know. Instructions stay in the file. Knowledge moves to a retrieval layer, where it costs nothing until it is used. With RDK's stacked retrieval most queries resolve before the model is involved at all, which means the material can be as detailed as you like without being paid for on every run.

The endpoint worth aiming at: a short instruction file you can read in a minute, and everything else retrievable.