What should an AI agent actually store in memory?

Things expensive to rediscover and not recorded elsewhere: constraints found the hard way, decisions with their rejected options, user preferences, environmental quirks. Not summaries of files, since the file is the source and will change.

Should agent memory be retrieved by recency or relevance?

Relevance. Recency copies conversation history and produces wrong context for tasks touching older parts of the system. Load a small durable tier unconditionally and retrieve everything else against the current task.

How do you stop agent memory from going stale?

Tag how each fact expires at write time, run a periodic review over the durable tier and frequently retrieved chunks, and prefer memory derived from source files so it cannot drift from what it describes.

What happens when two sessions write conflicting memories?

Enforce an explicit rule in code: newest wins, source file authoritative, or human resolves. Always retain the previous value, because a silent wrong overwrite is unrecoverable.

How to Design an Agent Memory System

An agent memory system needs four components: a write path that decides what is worth storing, a store with tiers for durable facts and bulk knowledge, a read path that retrieves by relevance rather than recency, and an expiry policy. Most designs fail on the write path, because storing everything produces a store that retrieves noise.

The four components

Sketch the system as a data flow rather than a feature list.

Write path. Something decides that a piece of information should outlive the session. This is a policy, and if you do not write it deliberately you get the default policy, which is store everything the agent found interesting. That default is why so many memory features degrade over a few weeks.

Store. Two tiers. A small durable set of curated facts, on the order of a few hundred to a few thousand tokens, that loads on every session. And a large indexed corpus, effectively unbounded, that is queried rather than loaded.

Read path. At the start of a task and at any point the agent needs background, retrieve from the corpus by relevance to the current query, and always load the durable tier.

Maintenance. Expiry, deduplication, and conflict resolution. This is the component everyone skips and the one that determines whether the system is still useful in month six.

The write path is where systems fail

Ask what qualifies. A useful test: would this still be true and still be worth knowing next month, and could it not be re-derived cheaply from the source?

That rules out most of what agents want to save. A summary of a file fails the test, because the file is the source and it will change. A conclusion drawn during debugging fails if the bug is now fixed. What passes is the class of thing that is expensive to rediscover and not written anywhere else: a constraint discovered the hard way, a decision and the options it rejected, a user preference, an environmental quirk.

There are three viable write policies and they can coexist. Explicit writes only when a human or the agent is told to remember something, which is precise and under-collects. Extractive runs a pass at the end of a session that pulls candidate facts, which collects more and needs a filter. Derived writes nothing itself, and treats the underlying files as the store, indexing them instead. The third is the most robust because the memory cannot drift from the source, and it should be your default for anything that lives in a document or a repository.

Store the reasoning, not the conclusion

"We use the queue rather than the log" is a fact that will confuse someone later. "We use the queue rather than the log because ordering per key was required and the log only guaranteed it per partition" is a fact that survives, because it tells a future reader when the decision stops applying.

The read path: relevance beats recency

The instinct is to load the most recent memories, because that is how conversation history works. It is the wrong default. An agent working on the billing module needs what is known about billing, whether that was written yesterday or last quarter.

So the read path is retrieval, not replay. Embed the current query, search the corpus, and return the top passages with enough surrounding context to be self contained. Load the durable tier unconditionally, because it is small and always applies.

Two refinements pay for themselves. First, retrieve on the task rather than on the last message, since the last message is often a follow up that lacks the nouns needed to search well. Second, allow the agent to re-query mid task, because what it needs to know at step six is not predictable at step one.

This is the layer RDK implements directly. Files from vaults, docs, and code are indexed as encrypted private chunks, and the agent searches those chunks before querying a model, which cuts token spend 80 to 90 percent because the answer is retrieved instead of regenerated. In a stacked configuration private retrieval answers 40 to 65 percent of queries, the public network adds 15 to 20 percent, and the model handles the last 5 to 10 percent.

Chunk for self containment

A retrieved passage arrives with no surrounding document. If understanding it requires the heading three sections up, the chunk is wrong. Chunk on semantic boundaries and carry enough identifying context, such as the document title and section, into each chunk that a reader who sees only that chunk knows what it is about.

Expiry and conflict, the parts everyone skips

Every fact needs an answer to the question of how it stops being true.

Some facts expire by event: a decision is superseded, a service is decommissioned, a preference changes. Some expire by time: anything about the current sprint. Some never expire: why an old decision was made. Tag them at write time, because you will not be able to infer it later.

The cheapest useful mechanism is a review pass. Periodically surface the durable tier and any frequently retrieved chunks, and confirm or delete. It is unglamorous and it is the difference between a memory system and a growing pile of half true statements.

Conflicts arrive as soon as two sessions can write. Pick a rule and enforce it in code. Newest write wins is simple and loses history. Source file is authoritative resolves cleanly and is the reason derived memory is attractive. Human resolves is correct and does not scale. Whichever you choose, retain the previous value, since a wrong overwrite is otherwise silent and unrecoverable.

How to tell if it is working

Track two things. Retrieval precision, meaning how often the returned passages were actually used in the answer, and staleness rate, meaning how often a retrieved fact turned out to be no longer true. If precision is fine and staleness is climbing, your write path is too permissive rather than your index being wrong.

Frequently asked questions

What should an AI agent actually store in memory?: Things that are expensive to rediscover and not recorded anywhere else: constraints found the hard way, decisions along with the options they rejected, user preferences, and environmental quirks. Summaries of files should not be stored, since the file is the source and it will change. If it can be cheaply re-derived from a document, index the document instead.
Should agent memory be retrieved by recency or relevance?: Relevance. Recency ordering copies conversation history and produces the wrong context for any task touching older parts of the system. An agent working on billing needs what is known about billing regardless of when it was written. Load a small durable tier unconditionally, and retrieve everything else against the current task.
How do you stop agent memory from going stale?: Tag each fact with how it expires at the moment you write it, by event or by time, since you cannot infer it later. Run a periodic review over the durable tier and frequently retrieved chunks to confirm or delete. Better still, derive memory from source files so it cannot drift from the thing it describes.
What happens when two sessions write conflicting memories?: You need an explicit rule enforced in code. Newest write wins is simple but loses history. Treating the source file as authoritative resolves cleanly and is the strongest argument for derived memory. Human resolution is correct but does not scale. In every case keep the previous value, because a silent wrong overwrite is unrecoverable.

Key takeaways

Memory design is mostly a write problem. What you choose not to store determines retrieval quality more than any indexing decision.
Use two tiers with different rules: a small durable set of facts loaded every session, and a large indexed corpus queried on demand.
Retrieve by relevance, not recency. Recency ordering feels intuitive and produces the wrong context on any task that touches older parts of the system.
Every stored fact needs an expiry story. Stale memory is worse than absent memory because the agent follows it confidently.
Conflicts are inevitable once two sessions write. Decide upfront whether the newest write wins, the source of truth is the file, or a human resolves it.