How Embeddings Work
An embedding is a fixed-length vector of numbers produced by a model that maps text, or other data, into a space where items with similar meaning sit close together. Retrieval systems embed documents once, embed each query at search time, and return the documents whose vectors are nearest to the query vector, usually by cosine similarity.
From text to vector
An embedding model is a neural network trained so that its output vectors place related inputs near each other. Give it a sentence and it returns a list of numbers, often several hundred to a few thousand long. That list is the embedding.
The individual numbers mean nothing on their own. What carries information is geometry: the direction and distance between vectors. Two passages about configuring a database connection land near each other even if one says connection string and the other says DSN.
Training is what produces that behaviour. Models are commonly trained on pairs of texts that should match, such as a question and its answer or a title and its body, and pushed to place matching pairs closer than non-matching ones.
Tokens and length. The model tokenizes input and has a maximum input length. Text beyond that limit is truncated, which is one reason documents are split into chunks before embedding.
Pooling. The model produces an internal representation per token, then combines them into a single vector for the whole input. The vector therefore summarises the chunk; it does not preserve every detail inside it.
Embeddings beyond text
Models also embed images, audio, and code. Multimodal models place different types of input in a shared space, so a text query can retrieve an image. The same rules apply: one model for both sides, and similarity reflects what the model learned to treat as related.
How similarity search uses them
Indexing. Each chunk of a document is embedded once and stored alongside its text and metadata in a vector index.
Querying. The user's question is embedded with the same model at search time.
Ranking. The index returns the chunks whose vectors are closest to the query vector. Closeness is usually measured by cosine similarity, the angle between vectors, or by dot product, which equals cosine similarity when vectors are normalised to unit length.
Approximate search. Comparing a query against every stored vector is slow at scale, so vector indexes use approximate nearest neighbour algorithms that trade a small amount of recall for large speed gains.
Same model on both sides. Query and document vectors must come from the same model and version. Vectors from different models live in unrelated spaces. Changing embedding models means re-embedding the corpus.
Embedding once and retrieving many times is also why retrieval is cheap per query: the expensive model call over the corpus is paid at indexing time, not every time someone asks.
Dense and sparse together
Dense embeddings capture meaning. Sparse keyword methods such as BM25 capture exact terms. Many production systems run both and merge the results, often called hybrid search, because each catches what the other misses.
Where embeddings fail
Exact identifiers. Error codes, part numbers, function names, and version strings are often embedded close to similar-looking but different identifiers. Keyword search or metadata filters handle these better.
Negation and qualifiers. A passage saying a setting is not supported and one saying it is supported can embed very close together, because they share nearly everything except the word that matters.
Numbers and dates. Embeddings represent that a passage contains figures, not which figures are correct for the question.
Long, mixed chunks. A chunk covering several topics produces a vector that sits between them and matches none of them strongly. Chunking strategy directly affects retrieval quality.
Domain vocabulary. A general model may not separate terms that are distinct in a specialised field. Domain-tuned models or hybrid search help.
Cross-lingual gaps. Unless a model was trained for multiple languages, a query in one language may not match documents in another.
The practical response is to treat embedding similarity as a strong first filter, then add keyword matching, metadata filters, and re-ranking where precision matters.
In RDK, chunks from indexed vaults are embedded and searched this way before an agent calls a model, so repeated or reference-heavy questions are answered from retrieved chunks rather than regenerated.
Re-ranking
A re-ranker reads the query and each candidate chunk together and scores how well the chunk answers the query. It is slower than vector search, so it runs only on the top results, and it often corrects the ordering mistakes that embedding similarity makes.
Frequently asked questions
- How do embeddings work?
- An embedding model converts text into a fixed-length vector so that passages with similar meaning have nearby vectors. Retrieval systems embed documents once, embed each query at search time with the same model, and return the documents whose vectors are closest, usually measured by cosine similarity or dot product.
- What are embeddings in AI?
- Numerical vector representations of data, such as text, images, or code, produced by a trained model. Their geometry reflects meaning: related items are close together. They power semantic search, retrieval augmented generation, clustering, recommendations, and duplicate detection across many AI systems.
- Can you mix embeddings from different models?
- No. Each model produces vectors in its own space, so a query embedded with one model cannot be meaningfully compared against documents embedded with another, even if the vector lengths match. Switching models, or model versions, requires re-embedding the entire corpus.
- Why do embeddings miss exact matches like error codes?
- Embeddings capture overall meaning and treat similar-looking identifiers as close. An error code or function name that differs by one character can embed near the wrong one. Keyword search, metadata filters, or hybrid search that combines both approaches handle exact identifiers far more reliably.