AI Memory vs Context Window
The context window is the maximum number of tokens a model can process in a single call, covering instructions, history, retrieved text, and output. Memory is information stored outside the model that persists across calls, such as facts, preferences, and past work, and is retrieved into the context window when relevant. Context is working space; memory is storage.
What the context window is
A model reads a fixed maximum number of tokens per call. That budget holds the system prompt, tool definitions, conversation history, retrieved documents, and the response. Nothing outside it exists to the model. When a conversation grows beyond the window, older content must be dropped, summarised, or retrieved selectively.
What memory is
Memory is storage outside the model. Short-term memory might be a running conversation summary. Long-term memory might be a store of facts, decisions, notes, and past outputs, often indexed with embeddings. An agent writes to memory as it works and reads from it by retrieving relevant items into the next call's context.
Cost differences
Everything in the context window is billed as input tokens on every call. Keeping a long history or full documents in context repeats that cost each time. Memory costs storage and retrieval, which are cheap by comparison, and only the relevant pieces enter the context.
Quality differences
Very long contexts can dilute attention: models may miss details buried in the middle of large inputs. Memory with good retrieval sends a short, relevant context, which often improves accuracy as well as cost. Poor retrieval, however, sends the wrong memories, which is worse than none.
Do bigger context windows replace memory?
No. They raise the ceiling on what fits in one call, which helps with large single documents. But they do not persist anything between calls, they increase per-call cost when filled, and they do not organise knowledge. Agents that work over days or across projects still need memory.
Memory with RDK
RDK gives agents a persistent, searchable memory: vault notes and code indexed as encrypted private chunks, retrieved before the model is called. RetroDeck's stacked retrieval checks the private vault first, then the public network, and falls back to the LLM, with the private vault answering 40 to 65 percent of queries and the LLM 5 to 10 percent.
Types of agent memory
Working memory is the current context window. Episodic memory records what happened: past sessions, actions, and results. Semantic memory holds facts and knowledge, often as indexed documents or notes. Procedural memory holds how to do things, such as saved instructions or skills. Different stores suit each type, but all of them reach the model the same way: by being retrieved into context.
Managing the context budget
Treat the context window as a budget. Reserve space for instructions and output, then fill the rest with the most relevant retrieved memories, recent turns, and task data. Summarise older history instead of keeping it verbatim. Measure tokens per call over a session; if it grows every turn, memory is leaking into context.
Retrieval as durable memory
Retrieval is how memory reaches the model. A good retriever finds a few relevant items and places them in context with source labels. A weak one either floods the window or misses what matters, so retrieval quality sets the ceiling on memory quality.
| Aspect | Context window | Memory |
|---|---|---|
| Lifetime | One call | Persists across calls and sessions |
| Location | Inside the model's input | External store |
| Cost | Input tokens on every call | Storage plus retrieval, then only relevant tokens |
| Limit | Fixed maximum tokens | Effectively unbounded |
| Failure mode | Overflow, diluted attention | Retrieving the wrong items |
Frequently asked questions
- Is a context window the same as memory?
- No. The context window is the working space for a single call and is cleared afterwards. Memory is stored outside the model and persists. Memory becomes useful only when relevant parts are retrieved back into the context window. Agents need both: memory to keep knowledge, context to use it.
- Does a larger context window mean better memory?
- Not exactly. A larger window lets more fit into one call, but nothing persists after the call ends, and filling the window costs more tokens each time. Long-running agents still need external memory with retrieval. Very long inputs can also dilute attention.
- How do AI agents remember things between sessions?
- They write important information to an external store, such as notes, a database, or an indexed vault, and retrieve relevant entries into the context at the start of later calls. The quality of that retrieval decides how well the agent remembers.