Better Search, Better Agents: Reading Mixedbread's 36% to 98% Claim
Most agent accuracy failures are retrieval failures: the model never sees the passage that holds the answer. In an AI Engineer World's Fair talk, Mixedbread co-founder Aamir Shakir reported a hedge fund customer whose answer accuracy rose from 36% to 98% after switching search. The figure is Mixedbread's own and the question set is unpublished, so measure retrieval on your data.
What the talk claims
The talk, "How we taught agents to use good retrieval," is from AI Engineer World's Fair 2026 and is presented by Mixedbread's Hanna Lichtenberg, with co-founder Aamir Shakir opening it. Shakir makes three claims. Agents are smart enough, and most of the time the real problem is getting them to the right data. The usual stack of OCR, chunking, embeddings, and a vector database, which he calls the Frankenstein stack, loses information and leaves agents reading every document. And a customer's answer accuracy went from 36% to 98% after switching their search. The talk demonstrates multimodal, late-interaction retrieval live, including a query for Intel's foundry segment operating margin.
Coverage of the talk fills in some of the setup, checked November 5, 2026. The customer is described as a large hedge fund with more than a billion documents, which onboarded in about a week and a half; the system it replaced was chunking plus a generic embedding model and a vector database; answers were graded by human experts; and LLM spend is reported to have fallen threefold. What is still missing is the customer's name, how many questions were tested, and which model answered. Mixedbread's own site does not list the figure; its published accuracy numbers are benchmark results, not this customer story. Treat 36% to 98% as a vendor's account of one deployment. It is a useful prompt to examine your own retrieval, not a number to plan around.
Why retrieval is the lever
A model can only be as right as the context it is given. If the passage that answers the question is not retrieved, the agent has two options, both bad: answer from whatever it did retrieve, which produces confident wrong answers, or keep searching and reading, which burns tokens and time. Swapping in a stronger model fixes neither. It produces better-written answers from the same wrong context.
That is why retrieval changes can move accuracy far more than model changes. The ceiling on answer accuracy is roughly the rate at which the right evidence reaches the context window, and on hard document collections that rate can be low without anyone noticing, because the agent still returns fluent text.
Where the standard pipeline loses information
The talk's critique matches failure modes that show up in most document-heavy RAG systems.
Parsing
Earnings reports, filings, and slide decks put the answer in tables and charts. OCR and text extraction flatten tables, drop axis labels, and separate numbers from the headers that give them meaning. A figure that was readable on the page becomes an orphaned string.
Chunking
Fixed windows split a table from its caption or a definition from the clause that qualifies it. The chunk that gets retrieved is technically relevant and still missing the part that answers the question.
Single-vector compression
A standard dense embedding squeezes a whole chunk into one vector. Specific details, such as one quarter's gross margin in a page of numbers, get averaged into the overall gist, so a precise query matches a page about the right topic rather than the page with the right number.
What late interaction changes
Late-interaction retrieval, the approach popularised by ColBERT, keeps a vector per token, or per image patch in multimodal variants, instead of pooling them into one. At query time each query token is matched against its best-matching document token and the scores are summed. Fine-grained details survive because they are never averaged away. Multimodal versions embed the page image directly, so a chart or table is searched as it appears rather than through OCR output.
The trade-off is real. Storing many vectors per page multiplies index size, scoring is more expensive than a single dot product, and not every vector database stores multi-vectors natively. Mixedbread sells this approach, with a late-interaction embedding model and a multi-vector engine on its site, so the talk is also a product argument. The underlying point stands regardless of vendor: if your documents are visual or dense with figures, test a retriever that does not flatten them.
How to test a retrieval claim on your own data
Before believing any accuracy jump, including your own, measure retrieval and generation separately.
- Collect real questions. Fifty to a few hundred questions your users actually ask, each labelled with the passage or page that answers it.
- Score retrieval alone. For each question, check whether the labelled passage appears in the top k results. Recall at k and mean reciprocal rank tell you whether the evidence is reachable.
- Score answers with the model fixed. Run the same model and prompt over both retrieval setups, so the only variable is search.
- Categorise failures. Separate not retrieved, retrieved but ranked too low, retrieved but misread, and answer correct. Each category has a different fix.
- Count cost and latency. A retriever that wins on accuracy but triples index size or query time may still be the right choice, but decide that deliberately.
- Hold out a test set. Tune chunking and ranking on one split and report on another, or the improvement is partly overfitting.
A baseline of 36% on a task like this usually means many answers were not retrievable at all, through scanned pages, broken tables, or missing metadata. Fixing that kind of pipeline can produce dramatic gains. Improving an already decent pipeline rarely does, so compare against a fair baseline before attributing the jump to a particular technique.
Retrieval also decides cost
The talk's other complaint, agents reading every document, is a cost problem as much as an accuracy problem. Every page an agent reads because search failed is input tokens. Precise retrieval narrows what reaches the model, and retrieving stored answers instead of regenerating them is where RDK cuts token spend by 80 to 90 percent on repeated or reference-heavy work. Better search improves accuracy and the bill at the same time.
Frequently asked questions
- What did Mixedbread claim about 36% to 98% accuracy?
- In an AI Engineer World's Fair 2026 talk, Mixedbread co-founder Aamir Shakir shared a customer whose answer accuracy went from 36% to 98% after switching their search. Coverage describes the customer as a large hedge fund with more than a billion documents, the replaced system as chunking plus a generic embedding model and a vector database, and the grading as human expert review, but the customer is unnamed and the question set and answering model are unpublished, so it is a vendor-reported result rather than a reproducible benchmark.
- Is a jump from 36% to 98% typical when you improve search?
- No. Gains that large usually mean the original pipeline could not reach much of the content, for example because tables were flattened, pages were scanned images, or chunks split answers apart. Fixing a broken pipeline can transform accuracy. Moving from a reasonable retriever to a better one usually yields smaller, though still meaningful, improvements.
- What is late-interaction retrieval?
- Late interaction keeps a separate vector for each token or image patch in a document instead of compressing it into one. At query time, each query token is matched to its closest document token and the scores are combined. It preserves fine detail, such as a single figure in a table, at the cost of larger indexes and more expensive scoring.
- How do I know if my agent's errors come from retrieval or the model?
- Label a set of real questions with the passages that answer them, then check whether those passages appear in the retrieved top k. If they are missing, the problem is retrieval. If they are present and the answer is still wrong, look at ranking, context length, and the prompt. Keep the model fixed while you change search.