# Building a Smarter AI Agent with Neural RAG

Will Bryk

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/xnXqpUW_Kp8-building-smarter-ai-agent-neural-rag) · [Video](https://www.youtube.com/watch?v=xnXqpUW_Kp8)

## Building a Smarter AI Agent with Neural RAG

A query for shirts without stripes exposes a retrieval problem that grows more consequential when agents need precise results, rich context, and entire sets of matching entities.

Basic familiarity with language models and Python functions is helpful; the article introduces the retrieval concepts used in the demonstration.

### What should “shirts without stripes” return?

[0:44](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=44s)

What should a search engine return for a request as simple as “shirts without stripes”? The difficulty becomes clearer by tracing how search acquired its strengths. In Will Bryk’s 1998 Google example, a search for Australia finds documents containing that word, while **PageRank** ranks authority using the web’s link graph. Matching words finds candidates; the structure of the web helps order them.

By 2021, **GPT-3** offered a different interaction: a user could describe what they wanted in a paragraph, including linguistic subtleties, and receive a fitting response. Against that experience, Bryk’s Google example looked strange. A search for shirts without stripes returned images of striped shirts. The word *without* changed the intended result, but the results still matched the conspicuous nouns.

![Slide titled “2021 - GPT-3 felt magical, Google felt ancient,” with a text-generation interface on the left and Google images showing striped shirts on the right.](https://ai.engineer/api/talk-essay-media/fb7cf122b473f601eb400de5ab6fffaef8e2fbf26001faffcca332130d26a7ba)

GPT-3 output beside Google results for “shirts without stripes.”

Bryk attributed the failure to keyword comparison. His goal for [Exa](https://exa.ai/blog/announcing-exa) was to combine transformer language understanding with search: understand both the request and the documents well enough to return what the user meant. He describes committing at least ten years to the problem and having spent four years on it by the time of the talk.

### Applying transformers to search

[2:26](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=146s)

The initial investment went into research. Bryk recounts joining YC’s Summer 2021 batch, raising a couple of million dollars, and spending half on a GPU cluster. The team then spent a year and a half researching without talking to customers. He presents both choices with a warning against treating them as startup advice: founders should talk to users, even though this particular effort began with a difficult technical redesign.

The research question was whether GPT-3’s underlying approach—transformers and next-token prediction—could be applied to search. Bryk shows a W&B training plot and recalls a purple run as a breakthrough, one of several improvements emerging from experiments with datasets and transformer architectures. The plot marks a moment when the approach began to work; the talk does not specify the training objective or provide enough detail to reproduce that run.

### From an inverted index to document embeddings

[3:23](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=203s)

Consider an illustrative corpus of a trillion web documents. An **inverted index** maps words to the documents containing them: the entry for `brown`, for example, points to documents where that word occurs. At query time, a traditional retrieval system can use those mappings to find and rank candidates. Bryk explicitly calls this a simplification of Google’s operation, rather than a complete description of its architecture.

The alternative is to represent each document with an **embedding**, a list of numbers learned by a model. The intended representation captures more than vocabulary: it can encode ideas, meaning, and how people elsewhere on the web refer to the document. Bryk’s scaling argument is that these representations can become larger and more expressive. Invoking the bitter lesson, he proposes training transformers on increasing amounts of high-quality data to produce better document embeddings. This is a direction for scaling the system, not a measured guarantee that a larger vector always retrieves better results.

At search time, the same distinction appears on the query side:

| Stage | Keyword representation | Embedding representation |
|---|---|---|
| Index documents | Map words to documents | Encode documents as vectors |
| Represent the request | Use query words | Embed the whole request |
| Find candidates | Compare keyword evidence | Compare query and document vectors |

For `shirts without stripes`, embedding the whole request gives the representation a way to retain the relationship expressed by *without*. Bryk describes comparing that query embedding against the document embeddings; he does not specify the index structure or serving algorithm used to make that operation practical at web scale.

![The query “shirts without stripes” points toward a keyword table above and a stack of numerical embedding vectors below.](https://ai.engineer/api/talk-essay-media/80015fe5aa225b83a6160e99f2e623dce14341636d470c6caddea66baaa3288e)

Comparing a query against keyword and embedding indexes.

After a year and a half, Bryk reports that the resulting search engine returned shirts without stripes for that request. The example is deliberately simple, but the intended capability extends to paragraph-long queries whose constraints cannot be reduced cleanly to a few words.

### ChatGPT changes who needs search

[5:21](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=321s)

Bryk dates the search engine’s launch to November 2022 and recalls an enthusiastic response on Twitter. The company was then called Metaphor; the Exa rename followed in January 2024. In his account, ChatGPT arrived two weeks after the launch and immediately changed the question facing the team. If people could ask a language model for information directly, where would search fit?

Try asking **GPT-4**, using only its stored knowledge, for cool personal sites belonging to engineers in San Francisco. Bryk’s point is that the requested collection may simply not exist in the model’s weights. Fluent language generation does not make those weights a complete directory of the web.

Bryk estimates GPT-4 at a couple of trillion parameters and less than ten terabytes of weights, versus over a million terabytes of web documents. Those are his rough capacity estimates: the [GPT-4 Technical Report](https://arxiv.org/html/2303.08774v6) withholds model size. He adds that images and video increase the total further, describes the web as exabyte-scale, and connects that scale to the name Exa.

**Freshness is a separate reason to retrieve.** Even a model with enough capacity to retain a much larger collection would still face a web that changes continuously. The resulting retrieval-augmented generation loop is straightforward: the language model searches Exa for the personal sites, receives external results, and uses those results to compose its answer. Search supplies the information that generation alone cannot reliably supply.

### Precision, context, and comprehensive results

[7:38](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=458s)

Adding search to a language model leaves another design question: was that search engine built for the caller now using it? Bryk contrasts a human typing short queries, reading a few links, and caring about the results page with an AI system capable of issuing complex requests and rapidly analyzing much more information.

His criticism of simply attaching traditional search to AI is about the objective being optimized. A result that attracts a human click is not necessarily the most useful input to the next computational step. Here, *AI* means an AI product—the assistant and its surrounding workflow—not just a model considered in isolation. That product needs explicit control over what information comes back.

The first requirement is **precision about the requested entities**. Imagine a venture capitalist asking an assistant for startups working on something ambitious that feels like Bell Labs. The requested output is a list of startups. Pages that are interesting to click but do not identify matching companies create extra work for the agent. As the user refines the task, the agent might add New York City as a location constraint and then add further company qualities. Each search needs to preserve those constraints rather than loosely gesture at the topic.

The second requirement is **searching with accumulated context**. An assistant that has talked with someone all day may know enough to personalize restaurant or apartment recommendations. Its query could contain several paragraphs describing a software engineer’s preferences and asking for places that fit them. Compressing that context into a few keywords can throw away the information that makes the recommendation useful. Bryk contrasts Exa’s support for multiple paragraphs with what he tentatively describes as a few-dozen-keyword limit in Google; he does not establish an exact limit.

The third requirement is **comprehensive retrieval**. Bryk illustrates the difference with 10,000 pages: ten days of patient human processing versus three seconds for AI if parallelized. The comparison is illustrative, with no specified workload, hardware, or quality measurement. For investment research, the substantive requirement is finding all the companies in a space, rather than the 10 or 20 results he associates with ordinary search. He calls for APIs that can expose 1,000 or 10,000 results and apply semantic constraints such as every YC-funded startup working on AI. Large result counts make broader analysis possible, but the talk supplies no recall measurement establishing that every matching company was found.

### A larger space of possible queries

[11:45](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=705s)

Bryk maps these demands as subsets inside a larger circle of possible queries. Familiar keyword lookups occupy one region: Stripe pricing, someone’s GitHub page, or Taylor Swift’s boyfriend. After 2022, conversational models made another region commonplace: explain a concept to a five-year-old, or debug the code supplied in the prompt. Those requests can be useful without requiring web search at all.

Semantic retrieval adds requests such as finding people in San Francisco who know Assembly. Bryk credits Exa with helping introduce this style of query, qualifying that claim with his own awareness of the field. A further region contains requests such as finding every article that argues X and not Y, written by an author like Z. He describes Exa’s Websets product as beginning to handle such combinations.

![A large gray circle contains four blue circles with labeled query examples. A speaker inset partly obscures the lower-left annotation.](https://ai.engineer/api/talk-essay-media/51821d4c55aa1da6e4f0dd06ddf2c5ee13d7bc31916114dea2e1c3c43d226032)

The space of possible queries includes keyword, semantic, pure LLM, and super-complex requests.

That last region suggests treating the web as a database that can be filtered according to the user’s intent. The ambition is broader than any one search mode, and Bryk acknowledges that new kinds of requests still arrive that no available search engine can handle. The destination is one API capable of supplying an agent with keyword results, semantic matches, and increasingly complex collections of knowledge. Supporting every possible query remains the eventual goal.

### Giving the calling agent control

[14:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=864s)

The live demonstration begins in Exa’s search dashboard. A query produces both a list of results and code showing how to make the request. Controls expose result counts, date ranges, and domain restrictions. Bryk describes counts of ten, a hundred, or a thousand: many toggles may look excessive for a human interface, but they give a calling agent ways to express the actual retrieval task.

The dashboard also exposes neural and keyword search. These are the historical controls used in the demonstration. For implementation today, the [Search API reference](https://exa.ai/docs/reference/search), checked in August 2026, documents a default of 10 results, a public maximum of 100, and higher limits by arrangement. Its current contract should not be inferred from the older dashboard.

Bryk then opens `agent.py`, containing an agent called Mark that formats information as Markdown. The first request asks for a personal site belonging to an engineer in San Francisco who likes information retrieval. He selects neural search because the task describes a combination of attributes rather than naming a known destination. After a save and wrong-agent execution interruption, he describes the intended run: retrieve matching personal sites, then have Mark turn the results into Markdown.

The next request is `Will Bryk GitHub`. This time, Bryk explicitly switches to keyword search. The person and destination are already named, so the retrieval task is closer to a conventional lookup. The two demonstrations separate finding an unknown person who meets a description from finding a known person’s page.

### Neural discovery followed by keyword lookup

[16:52](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1012s)

The keyword demonstration returns information about Will Bryk’s GitHub. From there, the useful unit of design becomes a sequence of searches, rather than a single mode chosen for the entire agent. A system could use neural search to discover a list of entities and then use keyword search to resolve a particular page for each entity. The user’s request determines which retrieval operation is needed at each step.

Bryk demonstrates that composition with a GitHub agent he says he generated in one shot using **o3**. Its task is to find GitHub information for engineers in San Francisco who like information retrieval:

1. Run a neural search for the matching personal sites.
2. Extract people’s names from the results.
3. Run a keyword search for each name and GitHub.

The first step discovers the cohort; the later lookups resolve a destination for each discovered person.

In Python, that orchestration can be expressed with separate search and name-extraction operations. Keeping the operations separate makes the change in query type explicit:

```python
from collections.abc import Callable
from dataclasses import dataclass


@dataclass(frozen=True)
class SearchResult:
    title: str
    url: str
    text: str


Search = Callable[[str, int], list[SearchResult]]
ExtractNames = Callable[[list[SearchResult]], list[str]]


def find_github_profiles(
    neural_search: Search,
    extract_names: ExtractNames,
    keyword_search: Search,
    people_limit: int = 10,
) -> str:
    query = (
        "personal sites of engineers in San Francisco "
        "who like information retrieval"
    )
    personal_sites = neural_search(query, people_limit)
    names = extract_names(personal_sites)

    markdown = ["# GitHub search results"]
    for name in names:
        matches = keyword_search(f"{name} GitHub", 1)
        markdown.append(f"\n## {name}")
        if matches:
            match = matches[0]
            markdown.append(f"[{match.title}]({match.url})")
        else:
            markdown.append("No result returned.")

    return "\n".join(markdown)
```

The supplied operations handle retrieval and extraction; the function carries names from semantic discovery into explicit keyword requests and formats the returned links. A returned link is still a search result: this sequence contains no additional identity-verification step.

The live GitHub-agent run requests ten results; Bryk says a hundred or a thousand are available on an enterprise plan. He then shows the agent gathering GitHub information. The demonstration establishes the composed workflow, rather than an exhaustive enumeration of every engineer who fits the description.

### Moving the research loop behind an API

[18:07](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1087s)

The closing announcement moves that orchestration one level higher. Bryk says Exa launched a Research endpoint that day: instead of asking the caller to coordinate each individual retrieval, the service would perform searches and language-model calls in the background to produce a report or structured output. He describes it as a deep research API and calls it state of the art, without presenting an evaluation. The architectural shift is from an agent requesting individual pieces of evidence to an agent requesting a research result, with the search-and-synthesis loop running behind the API.

## Resources

- [Announcing Exa](https://exa.ai/blog/announcing-exa) — Will Bryk explains the Metaphor-to-Exa rename and the company's approach to semantic retrieval, long queries, and extracting relevant page passages.
- [Exa Search API reference](https://exa.ai/docs/reference/search) — Current request parameters, result limits, filters, and content-extraction options for building applications with Exa Search.
- [GPT-4 Technical Report](https://arxiv.org/html/2303.08774v6) — OpenAI's report on GPT-4 capabilities and limitations, including which architectural details were not disclosed.

## Slides

### 1998: Google felt magical

An Australia search illustrates Google’s early promise: finding web documents and ranking results by authority using PageRank and the web’s graph structure.

[0:59](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=59s) [A slide titled “1998 - Google felt magical” shows Google search results for Australia.](https://assets.ai.engineer/sites/aiecode2025/sha256/55391197f67066483790bf9326e8a64614f311af3120878d3f21281986eb3b43.webp)

1998 - Google felt magical

Discover Australia
www.australia.com
Find out more with a week vacation in Australia. Go Walkabout now!

Information About Australian Holidays & Travel - Tourism Australia
Find useful travel information to plan your holiday to Australia. Travel to Australia & discover things to do & places to go - Tourism Australia Official ...
Destinations - Things To Do - About Australia - Sydney
www.australia.com/ - Cached - Similar

Australia - Wikipedia, the free encyclopedia
Hyperlinked encyclopedia article covers the country's history, government and politics, geography, economy, demographics, language and culture.
Etymology - History - Politics - States and territories
en.wikipedia.org/wiki/Australia - Cached

Department of Immigration & Citizenship
Provides access to information about Australian visas, Australian citizenship.
[illegible]ov.au/ - Cached - Similar

Sponsored links
Live & Work in Australia
Australia seeks skilled workers.
Do you qualify? Free assessment.
migrationexpert.com/Australia

Australia Vacations
Easily Plan & Book Your Next Trip
With the Ultimate Free Online Guide
www.Away.com

Top Australia Vacations
Featured on Travel Channel & NBC TV
Virtuoso Agency: Free Upgrades
www.travelwizard.com/australia/

See your ad here »

Interface text:
australia
Search
About 646,000,000 results (0.43 seconds)
Advanced search
Sponsored link
AIE
Microsoft
smol ai

### 2021: GPT-3 and Google

GPT-3 generated reasons to start a succulent garden, while Google Images returned striped shirts for “shirts without stripes.” The speaker saw an opportunity to combine language understanding with search.

[1:15](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=75s) [A slide contrasts GPT-3 text generation with Google image results for shirts without stripes, which show striped shirts.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa24e186ffc59c16d1fb0a3dc77c01e1270614c28c8df4420cb68d907e5cc1d1.webp)

2021 - GPT-3 felt magical, Google felt ancient

Three reasons to start a succulent garden

1. Succulents are low-maintenance: They don't require much watering or fertilizing, and they can tolerate a wide range of light conditions.
2. Succulents are drought-tolerant: They're perfect for areas that receive little rainfall or irrigation.
3. Succulents add interest and variety to the landscape: With their wide range of shapes, sizes, and colors, they can provide a unique and eye-catching addition to any garden.

Interface text:
Mode
Engine: text-davinci-002
Temperature: 0.7
Maximum length: 256
Stop sequences
Enter sequence and press Tab
Submit

Google
shirts without stripes
All  Images  Shopping  Short videos  Videos  Forums  News
Bond Suits
A Brown Striped Shirt from Mason & Sons ...
Coldwater Creek · O...
Stripes & Stars Stai...
Amazon.com
Amazon.com: Men's Casual...
Appleseed's · In stock
Foxcroft Non-Iron Cla...
FASH STOP · In stock
No Collar Stripes Lo...
Glamood
Dolce & Gabbana st...
Coldwater Creek · In...
Twin Stripe No-Iron ...

AIE
Microsoft
smol ai

### GPU cluster

After raising a couple million dollars in 2021, the team spent half on a GPU cluster and a year and a half researching how to rebuild search using GPT-3’s transformer technology.

[2:30](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=150s) [A slide shows an open GPU server and a person standing beside a server rack.](https://assets.ai.engineer/sites/aiecode2025/sha256/481793b835595486d91076d82d66c133353fef45057b93b47d025f7654e067e9.webp)

NVIDIA
WARNING

AIE
aws
World's Fair

[2:47](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=167s) [An open GPU server chassis beside a photo of a person standing next to a server rack.](https://assets.ai.engineer/sites/aiecode2025/sha256/d0657d97726c9192fc0c4ce85605934f8b5eacc250e35b76cf899ef8044a7c89.webp)

NVIDIA
WARNING

AIE
aws
World's Fair

### Documents and keyword index

A keyword index maps words to the documents containing them. For example, “brown” appears in both documents; a search engine can compare query keywords against the index to find results.

[3:29](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=209s) [A stack of HTML documents labeled one trillion documents points to a keyword index mapping terms to two documents.](https://assets.ai.engineer/sites/aiecode2025/sha256/1ac4da8cdbe839958ded4e799d42a46cc101caf1ced555f334498b7f96f2ffa4.webp)

HTML
1 trillion documents

Term    Doc_1    Doc_2
Quick            X
The     X
brown   X        X
dog     X
dogs             X

keyword index

Interface text:
AIE
Microsoft
smol ai

[3:42](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=222s) [A collection of HTML documents points to a keyword index mapping terms to documents.](https://assets.ai.engineer/sites/aiecode2025/sha256/b623466e7d01a3607a45e13c376a4d167a96719f82557849d86ae24711c2f5be.webp)

HTML
1 trillion documents

Term    Doc_1    Doc_2
Quick            X
The     X
brown   X        X
dog     X
dogs             X

keyword index

Interface text:
AIE
Microsoft
smol ai

### Keyword and embedding indexes

The diagram maps 1 trillion documents to both keyword and embedding indexes. Each embedding is a list of numbers that can capture a document’s meaning as well as its words.

[4:06](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=246s) [HTML documents point to both a keyword index and an embedding index represented by a list of numbers.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c89911c312eb949466a8584b00eff1b5e6a760217514a85abafbeab9640f620.webp)

HTML
1 trillion documents

Term    Doc_1    Doc_2
Quick            X
The     X
brown   X        X
dog     X
dogs             X

keyword index

[.43, -.52, .67, .89, -.41, .32, -.61, ... -.75]
embedding index

AIE
Microsoft
smol ai

### Searching with embeddings

For “shirts without stripes,” Exa embeds the query and compares it with embeddings of a trillion documents, instead of relying on keyword comparison.

[4:49](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=289s) [An example query points to keyword and embedding indexes, with the embedding index circled.](https://assets.ai.engineer/sites/aiecode2025/sha256/e95b314f380d9c94c458ca9efccff7027921aef0d1cf1f105440ded5590212ee.webp)

Term    Doc_1    Doc_2
Quick   |        | X
The     | X      |
brown   | X      | X
dog     | X      |
dogs    |        | X

keyword index

“shirts without stripes”

[.43, -.52, .67, .89, -.41, .32, -.61, ... -.75]

embedding index

Interface text:
AIE
Microsoft
smol ai

### Exa search results

Exa returns shirt listings for “shirts without stripes.” The speaker says the results have no stripes and that the search engine can also handle longer, more complex queries.

[5:00](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=300s) [Exa shows search results for shirts without stripes.](https://assets.ai.engineer/sites/aiecode2025/sha256/d24e72bec7748cb130a4b21d80985cccb734fcab1b1ef6064364cb035ebc6f85.webp)

exa

shirts without stripes

Standard White Shirt
clan.com/standard-white-shirt | Over 13 years ago Find similar

No-iron white cotton Shirt
hockerty.com/en/men/shirts/110371-no-iron-white-cotton-shirt | 11 months ago Find similar

White Mainstay Cotton Shirt - Buck Mason- Modern American Classics
buckmason.com/products/white-mainstay-cotton-shirt?sscid=21k8_a... | Over 12 years ago Find similar

Premium no-iron white cotton Shirt
hockerty.com/en-us/men/shirts/85990-premium-no-iron-white-cotton-s... | 11 months ago Find similar

White non-iron dress Shirt
hockerty.com/en-us/men/shirts/85992-premium-no-iron-white-cotton-s... | 11 months ago Find similar

Premium no-iron white cotton rounded collar Shirt
hockerty.com/en-us/men/shirts/419886-premium-no-iron-white-cotton-r... | 11 months ago Find similar

Interface text:
Category
All
Company
News
Paper
Github
Blog post
Personal site
Linkedin profile
Remove listicles
[illegible]tom category
Neural Search
Any time
Domain
Result Count
Compact
AIE
Microsoft
smol ai

### Search launch reactions

At Metaphor’s November 2022 launch, Siqi Chen compared the search experience to first using Google, while Joshua Meier highlighted its performance on scientific content.

[5:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=324s) [Three social posts discuss the launch and potential of AI-powered search.](https://assets.ai.engineer/sites/aiecode2025/sha256/7ada9ad876e1e5586d3f7d7a2cef503dc256c661b2cae899b2f06e9fd32a52f6.webp)

Siqi Chen @blader · Nov 10, 2022
This is the first time I've felt there's been a step change in Search experience since using Backrub / Google for the first time in the late 90s.

Nat Friedman @natfriedman · Nov 10, 2022
👀 LLM web search

Exa (prev. Metaphor) @ExaAILabs · Nov 10, 2022
metaphor.systems is now publicly available!

Metaphor is a search engine based on generative AI, the same sorts of techniques behind DALL-E 2 and GPT-3
...
Show more

Joshua Meier @joshim5 · Nov 10, 2022
This is the future of search. New ways to express what you're seeking. Enabled by powerful language models from the team @metaphorsystems. And it works particularly well for scientific content! 🤓

Interface text:
AIE
aws

### ChatGPT launch announcement

Two weeks after the search launch, Sam Altman announced ChatGPT and invited people to try it.

[5:30](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=330s) [A social post announces the launch of ChatGPT in November 2022.](https://assets.ai.engineer/sites/aiecode2025/sha256/c2a5a5222bf50fec75843586aec8b6559b817dc456953d8eeb26f8bf2baeddca.webp)

Sam Altman
@sama

today we launched ChatGPT. try talking with it here:

chat.openai.com

2:38 PM · Nov 30, 2022

Interface text:
1.1K · 7.1K · 29K · 7.3K
AIE
Microsoft
smol ai

### The Exa team’s reaction

SpongeBob lying awake depicts Exa’s reaction to ChatGPT: the team wondered whether search still had a role, then concluded it did because LLMs don’t know everything on the web.

[5:48](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=348s) [SpongeBob lies awake in bed.](https://assets.ai.engineer/sites/aiecode2025/sha256/0918a4c4a39da2bab7cc3051e617aeba7a8f72e327d930c03abac11074da209d.webp)

Interface text:
AIE
Microsoft
smol ai

### An LLM without search

Asked for San Francisco engineers’ personal sites, GPT-4 says it cannot browse in real time and suggests places to look. The example shows why the speaker sees a role for search alongside LLMs.

[6:04](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=364s) [A GPT-4 response says it cannot browse in real time to find engineers’ personal websites.](https://assets.ai.engineer/sites/aiecode2025/sha256/ae4d5d414a72fa66aa078792add1eb1f6e4b19fce99cf2a9310955e2d766b1c4.webp)

LLM

find me cool personal sites of engineers in SF

I can't browse the internet in real-time, so I can't provide a list of current personal sites for engineers in San Francisco. However, you might want to check out sites like GitHub or personal blogs to discover cool engineers in your area. San Francisco has a vibrant tech community, and many engineers share their work or thoughts online. Networking events, hackathons, and tech meetups are other good places to meet engineers who might have interesting personal sites. Platforms like Medium, Dev.to, or even LinkedIn could also be helpful for discovering such individuals.

Interface text:
GPT-4
AIE
Microsoft
smol ai

### GPT-4 and the internet

The speaker contrasts an estimated under 10 terabytes of GPT-4 weights with a far larger internet, arguing that the web’s size and continual updates make search useful for LLMs.

[6:18](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=378s) [A small network and a large network illustrate the slide’s comparison: GPT-4 under 10 terabytes and the internet over one million terabytes.](https://assets.ai.engineer/sites/aiecode2025/sha256/a33ae9984b8c25fbaf929f1ae086fee1b6260f701534a6982d3b55e1f0bc28c4.webp)

GPT-4: < 10 terabytes
The internet: >>1 million terabytes

Interface text:
AIE
Microsoft
smol ai

### LLM + Exa

An LLM searches Exa for personal sites of engineers in San Francisco, then uses the results to answer the user with a list of sites.

[7:11](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=431s) [A search-assisted LLM response lists personal websites of San Francisco engineers.](https://assets.ai.engineer/sites/aiecode2025/sha256/8c2e79da055856c1df69395e851cb488c61dcdb3df9b55e9c5be8825f57a9b41.webp)

LLM + Exa

find me cool personal sites of engineers in SF

Here are some cool personal websites of engineers based in San Francisco:
1. Saagar Jha
2. Terry Morse Software
3. Frank Chen - Published on 2022-01-01
4. Forrest Bourke - Published on 2014-01-14
5. Rex Feng
6. Patrick Lu - Published on 2015-01-01
7. Neil Panchal - Published on 2022-10-07
8. Evan Wallace - Published on 2010-01-01
9. Chris Maddox - Published on 2014-09-03
10. Ryan Panchadsaram

Feel free to explore these sites to get a sense of the diverse engineering talent in San Francisco.

Interface text:
Used Search
saagarjha.com
terrymorse.com
forrestbourke.com
AIE
Microsoft
smol ai

### Traditional search engines were built for humans

The speaker describes Google’s keyword-based search as designed for people who type queries and read a few links, illustrated by a person typing and a results page.

[7:37](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=457s) [A slide contrasts a person typing keywords with Google search results.](https://assets.ai.engineer/sites/aiecode2025/sha256/a7d9f369b82f7f6d4574d157379c673c2e43d86e8229e826accd2cafbeac4dab.webp)

Traditional search engines were built for humans

Humans want to type simple keywords to click a couple links
Google built a keyword based algorithm for humans

Humans searching
Google

Interface text:
AIE
aws
Search: startups working on something huge, feels like Bell Labs
News · Images · Reviews · Youtube · Videos · Shopping · Books
Reddit
What is the nearest modern equivalent to Bell Labs or Xerox Parc? : r/AskEngineers
Jan 22, 2016 — The more I learn about Bell Labs and Xerox Parc, the more I want to work there. But afaik both of them are now defunct, or no...
YCombinator News
Innovation and the Bell Labs Miracle | Hacker News
Feb 26, 2012 — This is not to say that most people working in startups are the same type of people who could have worked at Bell Labs - they aren't.

### A search engine built for AIs

The speaker argues that AIs can analyze information quickly, so they need complex queries and large amounts of knowledge. The slide pairs an AI search stream with Exa results.

[8:13](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=493s) [Slide contrasts complex AI queries and large amounts of knowledge with a search algorithm designed for AI, illustrated by terminal and Exa results.](https://assets.ai.engineer/sites/aiecode2025/sha256/9cf601c73618d54050891fe12a02c1b5e2249b60f08b09132b48b63d4e0eed34.webp)

What about a search engine built for AIs?

AIs want to use simple (struck through) complex queries to find a couple links (struck through) tons of knowledge

A different algorithm than Google is more optimal

AIs searching
[Terminal output, largely illegible]

Exa
"results": [
  {
    "url": "https://www.anandafuara.com/",
    "title": "Welcome to Ananda Fuara - Ananda [illegible]",
    "publishedDate": "2017-09-14T00:00:00",
    "author": null,
    "id": "[illegible]",
    "snippet": ""
  },
  {
    "url": "https://www.baiasf.com/",
    "title": "BAIA",
    "publishedDate": "2006-01-01T00:00:00",
    "author": null,
    "id": "[illegible]",
    "snippet": ""
  },
  {
    "url": "https://indochinevegan.com/",
    "title": "Contact",
    "publishedDate": "2015-06-11T00:00:00",
    "author": "IndoChine",
    [illegible]

AIE
Microsoft
smol ai

### AIs want precise, controllable information

For a hypothetical investor seeking startups like Bell Labs, Google shows discussions about modern equivalents, while Exa returns company pages. The speaker argues that AI products need results that match the requested information.

[9:02](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=542s) [Slide compares Google and Exa search results for startups working on ambitious projects like Bell Labs.](https://assets.ai.engineer/sites/aiecode2025/sha256/f550e2bab3ab5b377d2fb25a980348610a9cb372128f7b72d6bc742aa4d7a4cd.webp)

AIs want precise, controllable information

Google
startups working on something huge, feels like bell labs

Reddit · r/AskEngineers
What is the nearest modern equivalent to Bell Labs or ...
This includes government labs such as LLNL, Sandia, NASA’s labs (Ames, Dryden, Goddard, Langley, etc.), NRL, ARL, AFRL, places like NIST do a ...
43 answers · Top answer: Google, I’d say, but I wasn’t around back then. Google’s making a lot of advanc...
(Serious) Which company/workplace is the modern ...
What are the Bell Labs of today? : r/AskEngineers ...
More results from www.reddit.com

People also ask
What is similar to Bell Laboratories?
What replaced Bell Labs?
How prestigious is Bell Labs?
Why did Abe leave Bell Labs?

Quora
What is the modern day equivalent of Bell Labs?
[illegible] is no equivalent of Bell Labs today. Much of the work that Bell Labs did was pure research with no [illegible] motive.

Exa
startups working on something huge, feels like bell labs

PsiQuantum - Building the World’s First Useful Quantum Computer
psiquantum.com | Over 1 year ago | Find similar
Home - Calico
calicolabs.com | 12 months ago | Find similar
Altos Labs
altoslabs.com | Over 1 year ago | Find similar
Quaise Energy
quaise.energy | About 1 year ago | Find similar
Helion
helionenergy.com | About 1 year ago | Find similar
Atomic Semi
atomicsemi.com | Over 2 years ago | Find similar
Efficient Computer Corporation: Redefining energy-efficient computing.
efficient.computer | Over 1 year ago | Find similar
Ayar Labs: Solving Bandwidth and Power Bottlenecks with Optical I/O
ayarlabs.com | 11 months ago | Find similar

Interface text:
All · Images · Videos · Short videos · Forums · News · Shopping · More
Neural Search · Any time · Domain · Phrase · 20 Results · Compact
AIE · Microsoft · smol ai

### AIs want search with lots of context

A paper excerpt becomes a search for similar ideas. Google reports ignoring words beyond its 32-word limit and returns the original paper; Exa displays other papers.

[10:16](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=616s) [A paper excerpt is used as a query, with Google and Exa search results shown side by side.](https://assets.ai.engineer/sites/aiecode2025/sha256/d707902c5a560245e43ad560275947432ef445615a5b2b7af84b5182bf3c1857.webp)

AIs want search with lots of context

Language Models are Few-shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples…

Find me some papers with similar ideas

Google
Exa

Interface text:
Google query: performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
Paper with similar idea:
About 43,000 results (0.42 seconds)
“datasets” (and any subsequent words) was ignored because we limit queries to 32 words.
[2005.14165] Language Models are Few-Shot Learners
Language Models are Few-Shot Learners
Language Models are Few-Shot Learners

Exa results:
arxiv.org/abs/2208.01009 — over 1 year ago
Few-shot Adaptation Works with UnpredicTable Data
arxiv.org/abs/2104.08773v2 — over 2 years ago
Cross-Task Generalization via Natural Language Crowdsourcing Instruct...
arxiv.org/abs/2306.05539 — 7 months ago
Instruction Tuned Models are Quick Learners
arxiv.org/abs/2212.10711 — 12 months ago
Task Ambiguity in Humans and Language Models
arxiv.org/abs/2205.12393 — over 1 year ago
Fine-tuned Language Models are Continual Learners

AIE
Microsoft
smol ai

### The space of possible queries

The gray circle represents all possible queries. Four blue subsets illustrate keyword, semantic, pure LLM, and super-complex queries; the remaining space includes queries no one has thought of yet.

[11:43](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=703s) [A diagram places keyword, semantic, pure LLM and super-complex queries within a larger space of possible queries.](https://assets.ai.engineer/sites/aiecode2025/sha256/988df7be22f9f2be3060f24bd51a3cc0165514eb1ae060d1f8fb6f402453313d.webp)

The space of possible queries

Keyword
“stripe pricing”

Semantic
“People in SF who know assembly”

Pure LLM
“Explain this concept to me”

super-complex
“Find me every article that argues X and not Y from an author like Z”

Queries no one has thought to [illegible]t

Interface text:
AIE
Microsoft
smol ai

### One API for web information

Exa presents search, crawling, answer, and research endpoints as parts of one API. The speaker describes its ambition to handle keyword, semantic, and super-complex queries for AI systems.

[13:37](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=817s) [Exa dashboard slide showing search, crawling, answer, and research endpoints.](https://assets.ai.engineer/sites/aiecode2025/sha256/056cea88e7b4f0442384ad3faf7c90eb11e92d8fecf86a2b363b2ea6ef8546fd.webp)

One API to get any information from the web

Our endpoints

/search
Returns results and their contents

/crawling
Returns webpage contents
Space X

/answer
Fast, web-grounded answers
Who is the president of Brazil?
Luiz Inácio Lula da Silva is the current president of Brazil

/research
Long-running research, for reports and structured outputs
What is the latest CRISPR-based [illegible]
Searching...
Found 10 papers
Analyzing site 1

Get started
API Key
Get started in 5min
[redacted]
Usage
Past month
Docs
Exa MCP

Interface text:
Personal
Home
API Playground
Search
Crawling
Answer
Research BETA
Management
Usage
Billing
API Keys
Team Settings
Learn
Examples Library
Docs
Demos
API STATUS
AIE
Microsoft
smol ai

## Full transcript

[0:00](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=0s) [upbeat music] All right, so I was gonna give a live demo coding. [laughs]

[0:21](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=21s) No. [clapping] But, uh, well, I will, but I know you all are actually here to hear a cool story. So I'll tell you a story about web search built for AI, and then we do some coding at the end.

[0:32](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=32s) And the story will end with this slide, uh, one API to get any information from the web.

[0:39](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=39s) And you'll know what this means by the end. But the story starts in 1998,

[0:44](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=44s) and what you're looking at is the, the state-of-the-art in information retrieval in 1998. You type in a word, Australia, to this new search engine called Google, and it magically finds you all the documents that contain the word Australia from the web.

[0:58](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=58s) It's crazy. Um, and the, the big insight of Google was they had this PageRank algorithm, so, uh, the results are, are ranked by authority based on the graph structure of the web.

[1:08](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=68s) And this was a clever algorithm, and it was really cool. I was [REDACTED:age] at the time. So if I was conscious, I would have thought this was cool.

[1:15](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=75s) Um, okay, and now our story, our, now our story, uh, skips twenty-three years to 2021. Um, by this point, I was conscious, barely. And, uh, [laughs] uh, and I, I noticed that, you know, GPT-3 had recently come out, and it was this magical thing that you could input a whole paragraph explaining exactly what you want, uh, and it

[1:36](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=96s) would really understand the subtleties of your language and give you an output that exactly matched. Um, and it's hard to remember how magical this was, but it was really magical in 2021.

[1:45](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=105s) And at the same time, I noticed there was Google, which, you know, you type in a simple query like shirts without stripes, and it would give you shirts with stripes, which is crazy.

[1:54](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=114s) Uh, [laughs] it, like, it doesn't understand the word without, um, because it's doing a keyword comparison algorithm. And so I decided that for the next at least ten years, I'm gonna devote myself to building a search engine that combines the technology of GPT-3, uh, to, with a search engine to make a search engine that actually understands what you're

[2:13](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=133s) saying, uh, at a deep level and understands all the documents on the web at a deep level and gives you exactly what you asked for. This is a very big idea, and, and we're working on, we've worked on it for four years and, uh, a lot of progress.

[2:26](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=146s) But it would change the world if, if you actually solve this problem. And so in 2021, uh, we, we, we joined YC. Summer 2021, uh, we raised a couple million dollars, and we did what every YC startup should do.

[2:37](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=157s) We spent half of it on a GPU cluster. I'm joking. You shouldn't do that. [laughs]

[2:43](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=163s) Um, and, and then we also followed YC's advice, uh, where we didn't talk to any users or, or customers for a year and a half, and we just did research.

[2:52](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=172s) Um, again, you shouldn't do that. You should talk to users. But in our case, it made sense because we were trying to solve a really hard problem, which is like redesign search from scratch, um, using the, the same technology as GPT-3, this, like, next-token prediction idea with transformers.

[3:04](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=184s) What if you could apply the same thing, uh, to search? And this is actually one of our, uh, W&B training runs. Um, the purple one, I believe, is, was a breakthrough where it, like, really, it really, like, learned.

[3:13](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=193s) Uh, there was, like, a, a few breakthroughs along the way, uh, involving, like, random datasets and different, uh, transformer architectures that we were trying. And this purple one, like, really it starts to, like, work well.

[3:23](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=203s) Um, and the general idea we had was like, okay, so yeah, what is, what is a search engine? You have, like, a trillion documents on the web. Um, and traditional search engines, uh, on a very high level will create, like, a keyword index of those documents.

[3:35](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=215s) So for each document, you, you are, you say, you ask what are the words in those document, and you create this big inverted index where you map from, like, words like brown to all the documents that contain that word.

[3:45](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=225s) Um, and then at search time, you know, when a shirts without stripes comes in, you do some crazy keyword, uh, comparison algorithm and get the top results. That's obviously a simplification of what Google does, but at a fundamental level, it's doing, it's like a keyword comparison.

[4:00](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=240s) But the idea was, like, what if you could actually... So with transformers, like, the, the big thing is, like, what if you could turn each document not into a set of keywords but into embeddings?

[4:08](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=248s) Uh, and these embeddings can be arbitrarily powerful, right? Like, it's a list of... An embedding is just a list of, of, of numbers, and, uh, it could represent lots of information, so.

[4:16](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=256s) An embedding, it doesn't just capture the words in the document, but also the meaning, the ideas in the, in the document and the way people refer to that document on the web.

[4:23](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=263s) And, you know, an embedding can be arbitrarily big, and so it, like, of course, uh, in the limit, it would just destroy keywords. And so you have this, like, arbitrarily powerful representation.

[4:32](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=272s) Um, and now the fundamental idea was just, like, the bitter lesson. What if we could, like, you know, train transformers to output embeddings for documents? And if we keep getting more and more data that's high quality, we could, uh, make a search engine that actually understands you.

[4:43](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=283s) And, um, the way it would work at inference, at, at search time is, like, a search comes in, a query comes in like shirts without stripes. Traditional search engines would use the above thing where they would do a, a very fancy keyword comparison and a bunch of other things.

[4:55](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=295s) Um, and then instead, we would just embed the shirts without stripes and compare it to the embeddings of all the trillion documents.

[5:02](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=302s) And, you know, after a year and a half, we actually had a new search engine that worked in a very different way. Uh, and you search shirts without stripes on Google-- I'm sorry, on Exa, [laughs] and you, um, you get a list of results that actually, uh, are not, do not have stripes.

[5:13](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=313s) Uh, it's a simple, uh, example, but, like, you could, uh, it could handle, like, more, way more complex queries, like paragraph-long queries.

[5:21](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=321s) And when we launched this in November 2022, we got a lot of excitement on Twitter. Um, this is a very new paradigm for search. You could do all sorts of interesting queries that you couldn't do before.

[5:30](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=330s) And then two weeks later, this happened. It was a small tweet. [laughs]

[5:36](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=336s) And, uh, this is a visual depiction of San Francisco at the time. [laughs] Um, you guys probably all re-remember this. [laughs]

[5:44](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=344s) And then this is a visual depiction of the Exa team at the time. [laughs] Because ChatGPT completely changed the way we interact with the world's information. You know, like, everyone can now use an LLM to just, like, talk, talk to their computer and, and get information.

[5:58](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=358s) And we were thinking, "Wait, is there even a role for search in this world? Like, these LLMs are so powerful." And then very quickly we realized, yes, there is a role because LLMs don't know everything on the web.

[6:08](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=368s) So for example, if you ask an LLM like GPT-4, "Find me cool personal sites of engineers in San Francisco," um, it'll, it, it can't. Like, it just doesn't have that in the weights, so it'll apologize, whatever.

[6:18](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=378s) Um-

[6:19](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=379s) And, you know, there's a very simple information theory argument here where it's like there literally isn't enough information in the weights of GPT-4 to store the whole web. GPT-4, call-- like, we don't know exactly how many, uh, parameters.

[6:29](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=389s) I think someone leaked it on YouTube once. But it's like, oh, you know, a couple trillion parameters. You could call it, like, less than ten terabytes, uh, in the weights of GPT-4.

[6:36](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=396s) And then the internet is, like, over a million terabytes, and that's just the documents on the web. Uh, there's also images and video, and it's way more. Um, actually the, the, the web-- if you look at-- I, I did a tweet recently about the, the size of the web, and it's r- it's in the exabyte range, um,

[6:51](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=411s) and our name is Exa. It's not a coincidence. Um, anyway, so, like, uh, uh, LLMs, uh, need to search the web just from this simple argument, and they're gonna need to do that for a long time, which, um, if you talk to ML researchers, they'll say the same thing.

[7:02](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=422s) It's just like, it, it's too hard. Also, the web is constantly updating. That's another problem. It's not just the size of the web, it's the constant updating-ness of the web that makes it very tricky.

[7:08](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=428s) So LLMs always will need search. That's great. Um, and so when you combine an LLM with a search engine like Exa, you can handle these, uh, queries. So, like, find me cool personal sites of engineers in SF.

[7:19](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=439s) Uh, the LLM will search Exa, get a list of personal sites, uh, and then, like, use that information to output the perfect thing for the user. You're all very familiar with this.

[7:28](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=448s) Like, LLMs plus search, it's obvious now, right? Like, everyone knows about it. But now let me tell you a secret about search that most people don't know. Um,

[7:38](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=458s) and the secret is that traditional search engines were not built for this world of AI. Traditional search engines were built for humans, uh, and humans are not-- are very different from AI.

[7:49](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=469s) Uh, so every search engine like Google, Bing, you name it, uh, was built in a different era for this kind of creature. Uh, this, this slow flesh human that's typing keywords and wants to read a few links and really cares about UI of the page and all these things.

[8:04](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=484s) Like, it's a lazy human. They type simple keywords. Google is great for this creature. Um, [chuckles] Google was optimized for this creature. It gives you exactly the kinds of things you would click on.

[8:14](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=494s) But AIs are very different. Um, this-- like, an AI can gobble up information like crazy. This is a much slowed down version of what our AIs, AIs probably feel like inside.

[8:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=504s) Uh, and so AIs are very different. They wanna use complex queries, not simple ones, to find not a couple links, but just tons of knowledge, as much knowledge as they could get, 'cause they actually have the patience to just analyze it all extremely fast.

[8:35](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=515s) And so the o- the, the search algorithm that's optimal for this type of creature is not the same [chuckles] algorithm that's optimal for the human. Like, that would be crazy if the same algorithm that was optimal for humans was optimal for, uh, AIs.

[8:48](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=528s) And so, like, all the t- a lot of the tools, the search tools that we're talking about these days on Twitter and stuff like that, they're still using, like, the old traditional search combined with AIs.

[8:57](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=537s) It's just not the right puzzle fit. Um, so Exa, we're really trying to think of, like, what is the right search engine for this AI world.

[9:04](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=544s) And so just a few examples, uh, we could dive deep into, um, to, to-- of how AIs are different. Well, AIs want precise, controllable information. So, oh, by the way, when I say AI, I'm usually-- I'm talking about, like, an AI product.

[9:16](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=556s) So imagine, like, in this case, like, a VC that's using, uh, an AI system to find a list of companies, uh, because they wanna invest. So, you know, they're looking for some-- what's the next big thing?

[9:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=564s) What's the next big thing that feels like Bell Labs? Well, when they tell their AI what they want, the AI will then go search a search engine, right? And if it searches a search engine like Google, it'll get a list of results that humans like to click on, but it's not very information dense, and it doesn't even

[9:37](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=577s) match what the person asked for. The, uh, or, sorry, what the AI asked for. The AI asked for startups working on something huge that feels like Bell Labs. It should get a list of startups.

[9:44](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=584s) It's kind of a crazy idea, but what if search engines actually returned exactly what you asked of them and not what you want to c- what Google knows you'll click on?

[9:52](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=592s) And so with AIs especially, they just want a search engine that returns exactly what they asked for. Because what's, what really the world's gonna look like is you're gonna interact with your AI agent, and you're gonna a-ask for something, and then it's gonna make tons of searches.

[10:03](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=603s) Like, okay, maybe they want startups working on something, like, similar to Bell La- Bell Labs. Maybe they want startups working only in New York City that have this quality and that quality.

[10:09](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=609s) And, and, uh, and it'll do all sorts of searches, and it just wants a search API that just does what it asks. And, and so you need a search engine like that.

[10:16](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=616s) So Exa's like that. Um, another difference between AIs and humans is AIs wanna search with lots of context. Again, if you're-- if you have an AI assistant and you talk to it all day, and then you ask for restaurants or apartments or, or what have you, uh, the AI has lots of context on you.

[10:30](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=630s) So it should be able to search with this, uh, large multi-paragraph thing saying like, you know, "My human is a software engineer, and it likes these types of things, and I like these types of things," and like, "Can you give me, uh, you know, restaurants that match those preferences?"

[10:42](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=642s) Uh, and so you need a search engine that could literally handle multiple paragraphs of text. But traditional search, like, search engines like Google were not meant to do that because humans would never type in multiple paragraphs 'cause they're too lazy.

[10:52](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=652s) And so Google was optimized for, like, simple keyword queries. So Google, I think, has, like, a, a few dozen keyword limit. Uh, whereas, uh, Exa can handle, like, multiple paragraphs o-of text.

[11:02](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=662s) Another big one where AIs are different than humans is AIs want comprehensive knowledge. Uh, like, if you give a human 10,000 links or 10,000 pages, it doesn't know what to do with that.

[11:11](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=671s) Like, it would take ten days of extreme patience to process all that. But AIs can do it in three seconds if it's parallelized, right? So if I'm an, a, a VC and I want a report on, like, all the companies in a space, I want literally all the companies, and there's a huge amount of value to getting

[11:26](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=686s) truly all of them and not just, like, the 10 or 20 that Google is able to find. And so you need a search engine that exposes the ability to return 1,000, 10,000, whatever it is, uh, and also has this semantic ability to, like, you know, when you say, like, every startup funded by YC working on AI, you

[11:40](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=700s) actually can get all of them. So, like, Google literally just can't do this at all.

[11:45](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=705s) Okay, I hope that through these examples, we see that the space of possible queries is actually, like, way larger than people realize. Uh, and u-until, like, 2022, we were kind of in this, like, top left blue world.

[11:57](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=717s) Uh, so the, the circle is, like, the space of possible queries, and the blues are, like, uh, you know, specific subsets of that space. And so, like, we were all in that top left corner of blue for a long time, where you could, you know, we could ha- uh, search engines could handle, like, uh, like, basic keyword

[12:12](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=732s) queries like Stripe pricing or, uh, someone's GitHub page or Taylor Swift's boyfriend or whatever it is. Uh, after 2022, everyone started to want the top right blue, uh, circle, where it was like, "Hey, uh, actually I wanna make queries like explain this concept to me like I'm a five-year-old," or, "Here's my code.

[12:28](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=748s) Can you, like, debug it?" The-this is a form of query. It doesn't require search, but it's a, it's a, another type of query that was introduced to the world in 2022.

[12:36](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=756s) And then, like- Uh, there's other types of queries like these semantic queries, like people in San Francisco who know Assembly. Uh, as far as I'm aware, E-Exa is like the-- I mean, Exa kinda like introduced this kind of query and, and, uh, and does re-really well on them, on, on those queries.

[12:49](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=769s) And then there's these like really complex queries like, find me every article that argues X and not Y from an author like Z. And we're starting to now have systems like, uh, Exa's like Websets product that could handle these things.

[12:59](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=779s) And I think this is actually a huge space because this like turns the web into like a database you could filter however you want, and that's really what AIs want.

[13:06](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=786s) They want this like full control database like query system that they could just get whatever they w- need for their user. And then there are the queries that no one has thought of yet.

[13:14](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=794s) Um, like every week, we get tons of queries and like, "Oh, wait, that's a really interesting type of query that, uh, [chuckles] that no search engine could do right now."

[13:21](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=801s) And, and eventually we'll try to, you know, handle all the, the queries that are possible. But there, there are so many new types of queries now because we have these AI systems and the stakes, like the, the expectations have just gotten way higher.

[13:33](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=813s) Okay, so now you c- we end our story, uh, with the same slide. One API to get any information from the web. So again, like Exa's trying to, if you go back, like handle not just like the keyword queries, but also the semantic queries, and also the super complex queries, and eventually all queries.

[13:50](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=830s) Um, we, we want one API that could like give these AI systems whatever knowledge they want. You have the AI and you have Exa providing, uh, the knowledge. Oh, I only have four minutes.

[13:59](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=839s) Okay. [chuckles] Um, okay. So that's-- Oh, let's see. Oop.

[14:06](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=846s) How do I go to a different part of my computer? Hmm.

[14:13](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=853s) Uh, if I change to the code editor, how do I do that? Let's see. What? Oh, it's there. Oh, but I can't see it. That's so weird. Use the mirror down here.

[14:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=864s) Oh, cool. Okay. [laughs] Okay, um- Are you using the mirror? There we go. Okay. Cool. Well, first of all, just, just very quick explanation of this is our, our, our search dashboard.

[14:35](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=875s) We could try different queries. I would just point out like in the search, uh, API endpoint, uh, you know, we exposed lots of different toggles. So first of all, you just try out a query and get, uh, it shows you the code, and it gets you, uh, a list of results.

[14:50](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=890s) Uh, and it exposes tons of different types of filters that you might wanna do. For example, like number of results, ten, a hundred, a thousand, whatever it is. Uh, you could have like date ranges or, you know, I only wanna search over these domains.

[15:00](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=900s) And it's a lot of toggles, but I think the, the point is actually you want the toggles 'cause your AI is actually gonna be calling this. You want a search engine that gives you full control.

[15:07](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=907s) Um, and we have like neural and keyword search, so you could try different ones. Um, okay, let me sw- quickly jump to the, the code. Okay, so I prepared this like code, uh, agent.py.

[15:19](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=919s) So we made this agent, uh, agent Mark, and Mark loves to make markdown out of things. Anything you give it, it'll make markdown-- Mark will make [chuckles] markdown. Uh, and so in this case, uh, we're going to here.

[15:32](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=932s) Well, I guess in this case, let's try, uh, this query, uh, personal site of engineer in San Francisco who likes information retrieval.

[15:42](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=942s) Uh, well, this is, this is the kind of a query that neural would be a lot better at. Whoa.

[15:48](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=948s) Okay. Have to save it. Oh, we're running the wrong agent.

[16:00](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=960s) Okay, so it's just, it's making a query to get like a list of personal sites of engineers in San Francisco who like information retrieval, and, and Mark, the agent, is just making a markdown output of that.

[16:08](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=968s) That's a very neural type query. You also might want, uh, do, uh, a different type of query, which is like a more keyword heavy one. Let's see, like, um ...

[16:26](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=986s) Will Bryk GitHub. Let me add my GitHub. So okay, so here I would want to make a keyword query. So you just change the keyword.

[16:40](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1000s) Search. So it's gonna get information from my, from my GitHub using keyword, uh, search, 'cause this is a very typical like Google like search that would work well, right?

[16:47](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1007s) Oh, God, I'm running this wrong one again. Okay.

[16:52](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1012s) Cool. That's information about Will, Will Bryk's GitHub. Um, and then, okay, so when you're actually building an agent, you're gonna be combining lots of different types of searches. So neural searches and keyword searches, uh, and all sorts of other searches that Exa exposes.

[17:04](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1024s) So like the right agent in the future is going to be this system that decides what type of search it needs, uh, for, uh, whatev-whatever the user, uh, says.

[17:12](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1032s) Like, it'll, it'll be like, "Oh, okay, I'm gonna make like a neural search to get a list of things, and then for each one I'm gonna do a keyword search," right?

[17:17](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1037s) You wanna give the, uh, agent like just full access to the world's information in however way it wants. Uh, not just keyword search, but also all these other things.

[17:26](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1046s) Um, and so here I one-shotted with o3 a GitHub agent which combines these two queries. So first it'll-- It, 'cause I wanna, you know, I wanna get the GitHub of every, uh, engineer in San Francisco who likes information retrieval.

[17:40](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1060s) Uh, so the agent will make, uh, a neural search to get a list of people, extract the names, and then search those o-using a keyword search to get their GitHubs.

[17:49](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1069s) And then if you run that ... Here, it's just getting ten results, but we could, you know, with Exa, we could do a hundred or a thousand if you're on an enterprise plan.

[17:59](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1079s) Uh, [chuckles] so now it's getting all the GitHub info.

[18:07](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1087s) Cool. So that's just a cl- an example. Um, and yeah, I mean, there are lots of other things that you could do with Exa, like, um, we actually just today r- uh, launched this research endpoint, um, where it will actually do like as much searches in the-- and LLM calls in the background to get you, uh, that

[18:24](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1104s) perfect report or that perfect structured output for the thing you asked for. So it's kinda like a deep research API, and a state-of-the-art deep resea- deep research API. Um, cool.

[18:33](https://www.youtube.com/watch?v=xnXqpUW_Kp8&t=1113s) That is the talk. I hope that was interesting. Thank you. [laughs] [upbeat music]
