How often should I update my embeddings?

As often as your data changes in ways that matter to answers. Code and operational data may need near real-time updates; documentation may be fine with hourly or daily syncs. Combine event-driven updates with periodic reconciliation to catch missed changes.

Do I need to re-embed everything when a document changes?

No, only the changed document. Delete its old chunks and insert new ones using stable IDs. Full re-embedding is needed when you change embedding models or chunking strategy, since vectors from different models or chunk boundaries are not comparable. Plan those rebuilds as migrations.

How do I remove deleted documents from a vector database?

Track source IDs on every chunk. When a source item is deleted, delete all chunks with that ID, either from a delete event or by comparing current source IDs with indexed IDs during reconciliation. Without this, deleted content keeps being retrieved.

How to Keep Embeddings in Sync When Source Data Changes

To keep embeddings in sync, detect changes at the source with content hashes, timestamps, or change data capture; re-chunk and re-embed only changed items; delete or tombstone chunks for removed content; use stable chunk IDs so updates replace rather than duplicate; re-embed everything when you change embedding models; and monitor freshness by comparing index state with the source.

Why sync matters

A retrieval index is a copy of source data. When documents are edited, records updated, or files deleted, the copy goes stale. Retrieval then returns old text, and the model answers from it with full confidence. In fast-changing domains, such as code, policies, and product data, stale indexes produce wrong answers quickly.

Step 1: detect changes

There are four common ways to find what changed. Most production systems use more than one.

Content hashes

Store a hash of each source item. On each sync, recompute and compare. Only items whose hash changed need processing. Works for files and documents of any kind.

Timestamps

Use modified times or updated_at columns to find recent changes. Cheap, but clocks and bulk updates can mislead, so combine with hashes where possible.

Change data capture

For databases, stream inserts, updates, and deletes from the database log. This gives near real-time updates without scanning tables.

Event hooks

Trigger re-indexing from application events, such as a document save or a git commit.

Step 2: use stable chunk IDs

Give every chunk an ID derived from its source, such as document ID plus position or heading path. When a document changes, delete all of its old chunks and insert the new ones under the document's ID. This avoids duplicates and orphaned chunks when boundaries shift after edits.

Step 3: re-embed incrementally

Process only changed items: re-chunk, embed, and upsert. Batch embedding calls for efficiency. Keep a queue so bursts of changes do not overwhelm the embedding service, and retry failures.

Step 4: handle deletes

Deleted source items must be removed from the index, or retrieval keeps returning them. With change data capture, delete events arrive directly. With file scanning, compare the set of current source IDs with indexed IDs and remove the difference. Tombstones, records marking deletion, help when deletes must propagate across systems.

Step 5: handle embedding model changes

Vectors from different embedding models are not comparable. When you change models, re-embed the entire corpus into a new index, validate retrieval quality, then switch queries over. Running old and new indexes side by side during the transition avoids downtime.

Step 6: keep metadata in sync too

Permissions, titles, tags, and timestamps change as well. If access rights change and the index is not updated, users may retrieve content they should no longer see. Sync metadata with the same pipeline, and enforce permissions at query time.

Step 7: monitor freshness

Track the lag between a source change and its appearance in the index, the number of items pending, failure counts, and a regular reconciliation that compares source and index counts and hashes. Alert when lag or mismatch exceeds a threshold.

Choosing a sync cadence

Real-time sync suits code and operational data where answers must reflect current state. Hourly or daily batches suit slower-moving documents. Many systems combine event-driven updates for critical sources with nightly reconciliation to catch anything missed.

Syncing vaults and code with RDK

RDK indexes vault files and codebases as encrypted private chunks. Keeping those chunks current matters, because agents retrieve them instead of regenerating answers, so a stale chunk becomes a wrong answer. File-level hashing and re-indexing changed files keeps the retrieval layer aligned with your notes and code.

Common pitfalls

Frequent problems include appending new chunks without deleting old ones, so outdated text competes with current text; forgetting deletes; mixing vectors from two embedding models in one index; ignoring permission changes; and running sync jobs with no monitoring, so failures go unnoticed until users report wrong answers.

Testing the sync pipeline

Write tests that create, edit, rename, and delete source items, then check the index reflects each change. Include edge cases such as large files, files that fail to parse, and bulk updates. Run them whenever the pipeline changes.

Costs of keeping fresh

Re-embedding costs compute or API fees, so incremental updates matter at scale. Hashing avoids re-embedding unchanged files after metadata-only edits. Batching reduces overhead. For very large corpora, prioritise freshness for the most-queried sources and sync long-tail content less often.

Versioning the index

Record on every chunk which embedding model, chunking settings, and pipeline version produced each chunk. When settings change, you can find and rebuild affected chunks, compare old and new results, and roll back quickly if retrieval quality drops after a change is deployed.

Frequently asked questions

How often should I update my embeddings?: As often as your data changes in ways that matter to answers. Code and operational data may need near real-time updates; documentation may be fine with hourly or daily syncs. Combine event-driven updates with periodic reconciliation to catch missed changes.
Do I need to re-embed everything when a document changes?: No, only the changed document. Delete its old chunks and insert new ones using stable IDs. Full re-embedding is needed when you change embedding models or chunking strategy, since vectors from different models or chunk boundaries are not comparable. Plan those rebuilds as migrations.
How do I remove deleted documents from a vector database?: Track source IDs on every chunk. When a source item is deleted, delete all chunks with that ID, either from a delete event or by comparing current source IDs with indexed IDs during reconciliation. Without this, deleted content keeps being retrieved.