How we taught agents to use good retrieval - Hanna Lichtenberg, Mixedbread AI
Hanna Lichtenberg · Aamir Shakir
AI Engineer World's Fair 2026 · 14:28
← All organizations
AI search and retrieval infrastructure
Mixedbread
Mixedbread builds search infrastructure that helps developers supply applications and AI agents with relevant information. Mixedbread Search handles ingestion and retrieval across text, PDFs, images, audio and video in more than 100 languages. Developers can upload data and retrieve parsed content through an API, with Python and TypeScript SDKs and integrations for coding assistants. Its embedding and reranking models support retrieval and relevance scoring, while Toast 1 is a specialized search agent that breaks queries into subqueries, gathers evidence and returns context. Toast 1 can also work with existing search backends.
Mixedbread began as an applied research lab; its founders include Aamir Shakir and Julius Lipp, the former CTO whose officer and employee roles ended in 2025. Its Wholembed encoder represents multiple media types in a shared embedding space, allocating more vectors to information-dense inputs. Its Silo retrieval engine filters candidates before fine-grained scoring and combines S3 storage with local caching. Together, these components make multi-vector retrieval practical without reducing each document to a single representation.
The business combines subscriptions with usage charges for indexing, searches, storage and agent tokens. Enterprise contracts offer dedicated infrastructure and bring-your-own-cloud options. In January 2026, the company reported more than one billion indexed documents and sustained throughput above 500 queries per second per search store—operational scale for its managed retrieval system.
Explore the recordings
The supplied Mixedbread archive contains one recording. Use it as a guide to retrieval and agent design for knowledge work; the arguments and benchmark results below are claims made in the recording, not independently verified findings or statements about Mixedbread’s products today.
Start with what makes knowledge work different
In Benjamin Clavié’s talk, begin with the contrast between coding and non-code work. He argues that coding benefits from durable identifiers, searchable file structure, and narrow tickets, while other knowledge tasks require reconstructing intent and judging whether information applies in context. This is a useful entry point for thinking about why a coding-agent workflow may need changes before serving broader research tasks.
Follow the retrieval experiments
Read the same recording for its account of retrieval as a performance constraint. On the leaderboard he calls Brass Compass, Clavié reports a leading configuration reaching 90.2% accuracy with 20% fewer tool calls. In the PDF benchmark he calls MatQA, he argues that multimodal search raises the ceiling imposed by lexical retrieval. Treat these as reported results within the talk’s experimental settings, rather than general performance guarantees.
Finish with research delegation and finite context
For orchestration ideas, follow Clavié’s discussion of specialized searcher agents returning short memos to an answering agent. He reports a 3.5-point accuracy improvement, which he characterizes as roughly a 40% reduction in the remaining oracle gap. His broader recommendation is to co-design search tools, agent training, and research roles around finite context, drawing on how human knowledge organizations divide work.
2 talks
Newest first
3 speakers at AIE
Affiliations reflect their AIE appearances, not necessarily current employment.
