What is the best embedding model?

There is no universal best. The right model retrieves correct chunks for your own queries at acceptable cost and latency. Use public benchmarks such as MTEB to shortlist, filter by language, domain, input length, hosting, and licence, then test candidates on labelled queries from your data.

The Massive Text Embedding Benchmark, a set of tasks and public leaderboards comparing embedding models on retrieval, classification, clustering, and more. It is useful for building a shortlist, but general benchmark results can differ from performance on your own documents and query patterns.

Do more embedding dimensions mean better retrieval?

Not necessarily. Larger vectors can hold more information, but they increase storage, memory, and search cost, and a well-trained smaller model can outperform a larger one. Some models support reduced dimensions with modest quality loss, which can be worthwhile at scale.

How do you evaluate an embedding model?

Label a set of real queries with the chunks that correctly answer them, index identical chunks with each candidate model, and measure recall at k and ranking metrics such as MRR or nDCG. Review failures by hand, and record cost and latency alongside quality.

How to Choose an Embedding Model

The best embedding model is the one that retrieves the right chunks for your queries at acceptable cost and latency. Shortlist models using public benchmarks such as MTEB, filter by language, domain, input length, dimensions, licence, and hosting, then compare them on a labelled set of your own queries using recall and ranking metrics.

The criteria that matter

Retrieval quality on your data. The deciding factor. A model that ranks well on general benchmarks can underperform on legal text, source code, or internal jargon.

Language coverage. If documents or queries span languages, use a model trained for multilingual retrieval.

Domain. Code, scientific, and legal content often benefit from models trained on that material, or from hybrid search with keyword matching.

Maximum input length. Text beyond the limit is truncated. Choose a model whose limit comfortably exceeds your chunk size.

Dimensions. Larger vectors can carry more information and cost more to store, keep in memory, and search. Some models support reduced dimensions with modest quality loss, which can matter at scale.

Latency and throughput. Query embedding happens on every search. Indexing throughput determines how long a full re-embed takes.

Cost. Per-token pricing for hosted APIs, or hardware and operations for self-hosted models.

Hosting and data control. Hosted APIs send your text to a provider. Self-hosted open models keep data local. For private material this can decide the question before quality does.

Licence. Open-weight models carry licences that may restrict commercial use.

Stability. A hosted model can be deprecated or changed. Pin versions and plan for re-embedding.

Open versus hosted in practice

Hosted models are fastest to adopt and scale without infrastructure work. Open-weight models run on your hardware, keep text private, and have predictable cost at volume, but need serving, monitoring, and upgrades. Teams indexing private vaults often start there for data control alone.

Using benchmarks without being misled

The Massive Text Embedding Benchmark, known as MTEB, publishes leaderboards comparing embedding models across retrieval, classification, clustering, and other tasks. It is a useful way to build a shortlist.

Three cautions apply.

Look at the retrieval tasks specifically. An average across all tasks can reward strengths you do not need.

Benchmark data is not your data. Public datasets differ from internal documentation, code, or support tickets in vocabulary, length, and structure.

Scores near the top are often close. Small differences on a leaderboard can reverse on a different corpus. Differences in cost, latency, and hosting are often larger and more certain than those quality gaps.

Use benchmarks to eliminate weak options and choose a handful to test, not to pick a winner.

What a shortlist should look like

Three to five candidates that meet your hard constraints on language, input length, hosting, and licence, spanning different sizes and costs. Including one smaller, cheaper model tells you whether the larger ones are worth what they cost on your data.

Testing on your own queries

Build an evaluation set. Collect real questions users or agents ask, and for each, mark which chunks correctly answer it. Even a few dozen labelled queries reveal large differences between models.

Index the same chunks with each candidate. Keep chunking identical so the model is the only variable.

Measure retrieval.

Recall at k: whether a correct chunk appears in the top k results.
MRR or nDCG: how high the correct chunks rank.
Failure review: read the queries each model gets wrong. The pattern of failures is often more useful than the score.

Include hard cases. Exact identifiers, negations, near-duplicate documents, and questions that use different vocabulary from the documents.

Measure the whole pipeline. If you add keyword search or re-ranking, evaluate the combination, since a slightly weaker embedding model plus a re-ranker can beat a stronger model alone.

Record cost and latency alongside quality. The best choice is the cheapest option that meets your quality bar, not the highest score.

Re-run the evaluation when documents change significantly, or before switching models, so decisions stay grounded in evidence.

Plan for the re-embed

Store raw chunk text with every vector and record the model name and version. When you change models, you re-embed from stored text rather than re-processing source files, and can run old and new indexes side by side until the evaluation confirms the switch.

Quantised vectors

Storing vectors at lower numeric precision cuts memory and speeds search, usually with a small recall loss. Evaluate quantised and full-precision indexes with the same query set before deciding, since the loss varies by model and data.

Frequently asked questions

What is the best embedding model?: There is no universal best. The right model retrieves correct chunks for your own queries at acceptable cost and latency. Use public benchmarks such as MTEB to shortlist, filter by language, domain, input length, hosting, and licence, then test candidates on labelled queries from your data.
What is MTEB?: The Massive Text Embedding Benchmark, a set of tasks and public leaderboards comparing embedding models on retrieval, classification, clustering, and more. It is useful for building a shortlist, but general benchmark results can differ from performance on your own documents and query patterns.
Do more embedding dimensions mean better retrieval?: Not necessarily. Larger vectors can hold more information, but they increase storage, memory, and search cost, and a well-trained smaller model can outperform a larger one. Some models support reduced dimensions with modest quality loss, which can be worthwhile at scale.
How do you evaluate an embedding model?: Label a set of real queries with the chunks that correctly answer them, index identical chunks with each candidate model, and measure recall at k and ranking metrics such as MRR or nDCG. Review failures by hand, and record cost and latency alongside quality.