Graph RAG vs Vector Search vs SQL for Agent Memory
Choose by question shape. Vector search finds passages by meaning and is the default for knowledge. SQL answers exact, filtered, and aggregate questions that similarity search handles badly. Graph traversal answers multi-hop relationship questions. Most working systems use vector search plus a relational store, and add a graph only when traversal is the actual need.
Match the store to the question
Most memory architecture arguments are really disagreements about which questions matter, conducted without anyone listing the questions.
So list them first. Take ten queries your agent actually needs to answer, written the way they will arrive. Then notice that they fall into three shapes.
Semantic. What do we know about this topic, how does this subsystem work, why was this decided. The asker does not know the exact wording of the answer, which is the entire reason keyword search fails here. This is vector search.
Exact and aggregate. Which items are in this state, how many since a date, what is the current value for this record. Precise, filterable, countable. This is SQL, and asking a vector index for it produces approximately correct answers that are worse than useless because they look right.
Relational and multi-hop. What depends on this service, which decisions were influenced by this constraint, who touched every component in this chain. This is traversal, and it is where a graph earns its keep.
The distribution of your ten questions decides the architecture, and for most agents it is dominated by the first shape with a meaningful tail of the second.
SQLite is a serious answer
A local relational file handles session state, entity records, counters, timestamps, and the durable facts an agent needs on every run, with transactions and no operational burden. Plenty of agent memory systems would be better and simpler as a vector index plus a SQLite file, and the reason they are not is that the combination sounds too plain to present.
What each one costs you
Vector search costs an embedding pipeline and reindexing whenever the embedding model changes, since vectors from different models are not comparable. Its failure mode is quiet: plausible passages that do not answer the question, which the agent then reasons over confidently.
Relational storage costs a schema, which means deciding structure in advance and migrating when you are wrong. Its failure mode is loud, which is a genuine advantage: a query that does not match the schema errors rather than returning something misleading.
Graph costs the most by a distance. Something must extract entities and relationships from unstructured content, and that extraction is itself a model-driven process with an error rate that compounds across hops. A wrong edge is invisible and produces confidently wrong traversals. Maintaining the graph as the underlying material changes is ongoing work that teams consistently underestimate at adoption time.
That is the honest case against reaching for graph first. It is powerful for genuine multi-hop questions and expensive for everything else.
What actually decides retrieval quality
Teams spend a week choosing a store and an hour on chunking, and then wonder why the results are mediocre. The ratio should be reversed.
A retrieved chunk arrives with no surrounding document. If understanding it requires the heading three sections above, it will confuse the agent regardless of how it was found. Chunk on semantic boundaries and carry identifying context into each chunk.
Curation matters as much. An index over everything you have ever written returns something for every query, much of it superseded, and superseded material retrieved confidently is worse than nothing.
And query formulation is the third lever nobody adjusts: retrieving on the current task rather than on the last message, since follow-ups often contain none of the nouns needed to search well.
RDK handles the layer underneath all of this. Files from local vaults, docs, and code are indexed as encrypted private chunks, and agents search those chunks before querying a model. Token spend drops 80 to 90 percent on repeated or reference-heavy work because the answer is retrieved instead of regenerated. Stacked retrieval sets the proportions: a private vault answers 40 to 65 percent of queries, the public network adds 15 to 20 percent, and the model handles the remaining 5 to 10 percent.
| Dimension | Vector search | Relational (SQL, SQLite) | Graph traversal |
|---|---|---|---|
| Answers best | What do we know about X | How many, which ones, since when | What connects to what, across hops |
| Query style | Similarity to a question | Exact filters and aggregates | Path and neighborhood traversal |
| Setup cost | Embedding pipeline and index | A schema | Entity and relationship extraction |
| Ongoing cost | Reindex when the model changes | Migrations | Keeping edges correct as content changes |
| Failure mode | Plausible but unhelpful passages | Errors, which is loud and useful | Confidently wrong traversals from bad edges |
| Good fit | Docs, notes, code, decisions | State, counters, entities, timestamps | Dependency and influence questions |
| Poor fit | Counting and precise filters | Open-ended meaning questions | Everything that is not multi-hop |
| Typical role | Primary knowledge layer | Durable facts and session state | Add only when traversal is the need |
Frequently asked questions
- Should I use Graph RAG for agent memory?
- Only when your real questions require traversing several relationships to answer. Graph costs entity and relationship extraction, which is model-driven and has an error rate that compounds across hops, plus ongoing maintenance as content changes. For questions about what is known on a topic, vector search answers better for far less.
- Is SQLite good enough for agent memory?
- For session state, entity records, counters, timestamps, and durable facts, yes, and it is frequently the right answer. It gives transactions and precise queries with no operational burden. Many agent memory systems would be simpler and better as a vector index plus a SQLite file than as the architecture they chose.
- Why does my vector search return irrelevant results?
- Usually chunking or curation rather than the store. A chunk that only makes sense under a heading it no longer carries will confuse the agent however it was retrieved, and an index containing drafts and superseded notes returns them with the same confidence as current material. Query formulation is the third common cause.
- Can one store handle everything?
- Not well. Similarity search answers meaning questions and gives approximately correct answers to counting and filtering questions, which is worse than an error because it looks right. Most working systems pair a vector index for knowledge with a relational store for exact facts, and add a graph only for genuine traversal needs.