AI Engineer World's Fair 2025

The State of AI-Powered Search and Retrieval

Frank Liu· Staff Product Manager, MongoDB12:35

Loading slides…

Read the talk

The State of AI-Powered Search and Retrieval

A shopping query like “My best friend is sick” should find get-well baskets. Making that work takes more than matching words: it takes retrieval that understands intent and respects constraints.

From a talk by Frank Liu

Before you start: Familiarity with LLMs and the idea of representing text as vectors will help; the article explains how those vectors fit into retrieval.

From matching words to understanding intent

Frank Liu approaches search from the embedding and reranking layer. His team at Voyage AI had joined MongoDB roughly three to four months before this talk. Voyage positioned its models around accuracy and cost for retrieval-augmented generation (RAG) and semantic search, with applications extending to classification and clustering. Liu listed the Voyage AI API, Azure, and AWS Marketplace as access points. Here, he moves beyond individual model evaluations to examine three application lessons and the capabilities search systems may need next.

AI-powered search finds related concepts even when the wording differs. Traditional methods such as TF-IDF and BM25 provide useful keyword retrieval, but conceptual relevance broadens what a system can find. The goal is not to discard keywords; it is to retrieve useful material when the query and the answer do not share them.

Consider the shopping query “My best friend is sick.” A useful result might be a get-well basket, even though the query never names that product. Merely matching “best friend” could produce generic gifts and miss the reason for buying one. Search needs to connect the situation to the user's likely intent. Liu adds limited reasoning and instruction following to this definition, returning to them later as directions for retrieval models.

Slide titled “What is AI-powered search?” with three points on related concepts, user intent using “my best friend is sick,” and reasoning or instruction-following.Open full source frame
AI-powered search finds related concepts, understands intent, and can perform reasoning or instruction-following.
0:250:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:25 · section reference included

Retrieval gives generation something to work with

RAG is the familiar application of this idea. Without relevant retrieved information, an LLM may hallucinate, refuse to answer, or produce a generic response. Retrieval supplies material from which it can construct a more grounded answer. Liu credits the RAG slides to Tengyu's talk and describes a short pipeline:

  1. Generate embeddings for the material to be searched.
  2. Search for information relevant to the prompt.
  3. Pass the retrieved information to the LLM as context for its response.

Embedding quality matters because it helps determine which information reaches the model. Liu estimates that 95–99% of the search systems he has encountered use embeddings in some form. That is an observation about his experience, with no sample size or survey method supplied. Text from PDFs, Word documents, Google Drive files, and PowerPoints can be represented in a shared vector space so that a query brings relevant documents toward the top of the results. A generator cannot make good use of a document that retrieval never supplies.

3:253:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:25 · section reference included

Choose models for the codebase, not the leaderboard

Chatting with a codebase makes the importance of domain fit concrete. Continue, whose code is largely open source, uses a RAG pipeline with reranking. Retrieval supplies candidate code and documentation; reranking helps order that material before it becomes context for an answer.

There is no single best embedding model or LLM for every application. Liu reports that Continue's evaluation found voyage-code-3 performed best for its application. He gives no numerical scores, dataset, or comparison protocol. The relevant requirement is clear, however: codebase chat depends on understanding code, documentation, and developer context. Evaluate model choices against those demands rather than assuming that a general-purpose result predicts performance on your repository.

4:474:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:47 · section reference included

Semantic relevance still needs structured constraints

In the second application, Liu draws attention to a blue box containing filtering and other structured inputs to search. The lesson is that embeddings alone are often insufficient. A document can be semantically relevant while failing a condition that determines whether the application may use it.

Take legal documents embedded with voyage-law-2. A query might concern the right legal topic but require documents from a particular state, only official documents, or documents containing a specific set of details. These requirements belong in a filtering stage. Liu says filtering typically happens inside the vector store, although some systems apply it afterward.

A small TypeScript example makes the distinction explicit. Suppose a semantic search has returned these three records, and the application requires official California documents:

typescript

type SearchHit = {
  id: string;
  state: string;
  official: boolean;
  score: number;
};

const hits: SearchHit[] = [
  { id: "doc-a", state: "NY", official: true, score: 0.94 },
  { id: "doc-b", state: "CA", official: false, score: 0.91 },
  { id: "doc-c", state: "CA", official: true, score: 0.87 },
];

const eligible = hits.filter(
  (hit) => hit.state === "CA" && hit.official
);

console.log(eligible.map((hit) => hit.id)); // ["doc-c"]

The illustrative scores rank semantic matches; the metadata determines eligibility. The highest-scoring result does not satisfy the state requirement, and the next result is unofficial. Only doc-c survives. This shows the post-retrieval variant of Liu's filtering stage: structured data adds constraints that similarity scores do not enforce.

5:536:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:53 · section reference included

An agent can search, reconsider, and search again

The third application changes retrieval from a single input-output operation into a feedback loop. An LLM can issue a search, inspect what comes back, and then expand or decompose the query. Retrieval becomes part of an ongoing reasoning process rather than a one-time preparation step.

Liu illustrates decomposition with an X/Twitter earnings request. His spoken example moves from Q4 to full-year 2024; the full-year interpretation explains the resulting plan: query Q1, Q2, Q3, and Q4 separately, sending the four searches to vector stores or databases. The example describes how to break up a request, not a retrieved set of financial results. Each subquery creates an opportunity to gather a distinct part of the information needed for the larger answer.

Calling 2025—and possibly 2026—the era of agents, Liu emphasizes another requirement: search must work well with conversational data. The diagram connects a human and an agent to both a knowledge vectorstore and a conversation vectorstore, with retrieval and return paths. It separates stored knowledge from conversation history while allowing both to inform the interaction. Liu notes that the diagram omits implementation details; its useful architectural point is that future searches can depend on earlier exchanges and retrieved results.

Diagram titled “Conversational/agentic retrieval” showing Human and Agent boxes connected to embedding, knowledge vectorstore, conversation vectorstore, and retrieval boxes by directional arrows.Open full source frame
Conversational retrieval connects a human and agent with knowledge and conversation vectorstores.
7:217:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:21 · section reference included

Tell the embedding what to emphasize

The next direction is instruction tuning, with reasoning as a secondary possibility. The baseline embedding interface takes a query or document and returns a vector. Liu envisions adding an instruction alongside that input to steer what the vector emphasizes. His forecast concerns richer steering and broader use of instruction-aware retrieval.

For example, a user might ask for documents that discuss one particular aspect of a topic in detail. Topic similarity alone can retrieve broad overviews; an additional instruction expresses the desired focus and depth. That differs from the earlier state or official-status filter: it changes the semantic preference the system should represent rather than testing a known metadata value. Liu presents this as a direction for search, without supplying a concrete reasoning implementation or a guarantee that an instruction will enforce every constraint.

10:3510:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:35 · section reference included

Bring the retrieval pipeline into the database

The final direction is what Liu calls an agent-native database, acknowledging the term's buzzword quality. Even a simplified retrieval architecture requires several components to work together. Voyage joining MongoDB makes the prospect of integrating those components particularly relevant to his team.

The proposed shift is from assembling a pipeline to using one data platform that performs its core retrieval operations:

OperationSeparate componentsProposed integrated platform
EmbeddingCall an embedding modelPlatform generates embeddings
Vector searchQuery a vector databasePlatform retrieves candidates
RerankingCall a rerankerPlatform reranks results
Query transformationCoordinate additional logicPossibly augment or decompose queries

The slide contrasts an embedding model, vector database, and reranker with a single MongoDB + Voyage AI box between query and results. Embedding and reranking are central to the proposal; query augmentation and decomposition are possibilities Liu explicitly leaves open.

Slide titled “Agent-native database” contrasts an embedding model, vector database, and reranker on the left with a single MongoDB + Voyage AI box connecting query and results on the right.Open full source frame
An agent-native database diagram contrasts separate retrieval components with MongoDB and Voyage AI.

For this June 2025 talk, the integrated platform is a forecast, not a deployment recipe. Liu expects more consolidation during that year and the next. The substantive promise is to make the database responsible for more of the retrieval work—representing information, finding candidates, and ordering results—so application builders have fewer separate components to coordinate.

11:2711:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:27 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…