# Effective agent design patterns in production

Laurie Voss, LlamaIndex

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/72XxWkd8Jrk-effective-agent-design-patterns-in-production) · [Video](https://www.youtube.com/watch?v=72XxWkd8Jrk)

## Effective agent design patterns in production

Build agents around messy inputs, selective retrieval, and explicit control flow, then combine chaining, routing, parallel work, and feedback to handle more demanding tasks.

Familiarity with LLM prompts, Python functions, type annotations, and async calls will help with the workflow example.

### Getting from documents to an application

[0:16](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=16s)

How do you get from a collection of documents to a useful generative AI application without spending all your time building plumbing? Laurie Voss, VP of Developer Relations at LlamaIndex, opens with the pieces his team provides for that job. The framework supports Python and TypeScript, with agents as a particular focus; the surrounding services handle document parsing, retrieval, and integrations.

**LlamaParse** handles complicated formats such as PDFs, Word documents, and PowerPoints. Parsing matters because an agent can only reason effectively about information it can understand. Voss attributes better agent quality to LlamaParse making those documents more intelligible to an LLM than unspecified open source parsers, though he provides no measured comparison.

In the talk’s product lineup, **LlamaCloud** is the paid service for putting documents in and getting a retrieval endpoint out, available as SaaS or deployed into a customer’s private cloud. [LlamaHub](https://llamahub.ai/) supplies the integration registry: adapters for sources such as Notion, Slack, and databases, connections to vector stores and LLMs, and prebuilt agent tools. Voss reports that LlamaIndex supported 400 models across 80 providers at the time of the talk, including local models such as **Llama 3**. The framework’s practical promise is to remove boilerplate so developers can spend their limited time on the business problem.

![LlamaHub slide showing Data Loaders, LLMs, Embedding Models, Agent Tools, Vector Stores, and Pre-built strategies.](https://ai.engineer/api/talk-essay-media/7a743ff065f3782368d7cf89f0349da603e116d4d65d3fce68a4d46319870091)

LlamaHub’s integration categories.

### Use agent flexibility where the inputs are messy

[2:55](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=175s)

Retrieval-augmented generation and agents are the two application categories Voss highlights. His working definition of an **agent** is semi-autonomous software that uses tools to achieve a goal without requiring the programmer to prescribe every step. Tools can retrieve information or take action; the LLM decides which ones to use. That decision-making power is useful when the application must deal with something unexpected or unknown, especially unstructured inputs.

Voss’s design heuristic is to look for tasks that turn a large body of text into a smaller, useful output. Interpreting a contract, processing an invoice, applying regulations, and summarizing documents all fit. The result might be a calendar event, a decision, a report, or an answer. He favors this reduction of existing material over asking a model to expand a small prompt into a large body of text.

**The application does not have to be a chatbot.** An LLM can convert messy input into structured data, make a decision about it, and feed the result into ordinary software. Voss characterizes chat as a 2023 usage pattern and sees a larger opportunity in embedding these capabilities into existing applications. The useful boundary is between the unstructured material the model interprets and the structured information the rest of the program consumes.

### Retrieve the context the model needs

[5:36](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=336s)

Unless the application is extremely generic, the model needs context about the company, domain, or particular problem. Asking questions without supplying that information is not enough. But a company may have far more data than it is practical to send with every request. This is the motivation for **RAG**.

The retrieval path has three stages:

1. **Embed the corpus.** Represent source content as vectors in a searchable vector space.
2. **Embed and retrieve for the query.** Represent the question in the same space and search for nearby content.
3. **Generate with the retrieved context.** Give the model the relevant material rather than the whole corpus, then ask it to answer.

The slide makes the last step concrete with a cat-location question and the answer “The Mat.” Retrieval selects the context that makes the answer possible; generation uses that context to respond.

![A diagram titled “RAG = infinite context” shows Embed, Retrieve, and Query stages, vector-space plots, and a cat-location question answered with “The Mat.”](https://ai.engineer/api/talk-essay-media/ec9079c8773d4b4cb1e813d7522e9d9bd6aad88402df105f39c374f22a8d863f)

RAG illustrated through embedding, retrieval, and answering with context.

Voss argues that larger context windows do not remove the value of retrieval: supplying less data reduces the amount the model must process, while more specific context can help it answer the question. His cost, speed, and quality claims here are a design rationale rather than a measured comparison. An agent can expose this retrieval-and-answering operation as one of its tools.

### Give retrieval a control loop

[7:12](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=432s)

**Naive top-K RAG** follows a fixed path: receive a query, retrieve the most relevant context, and send it to the LLM. That path cannot itself decide that the question needs to be decomposed or that an extraction should be repeated. An agent adds those decisions around retrieval.

The useful checks are specific:

- **Decompose the question.** Could a complicated question be answered more easily as several simpler questions?
- **Retry extraction.** Does the extracted data make sense, or should the system try again?
- **Review the answer.** Did the response answer the question sensibly, or does it need another pass?

Voss reports improvements in speed and, especially, accuracy from agent layers in production RAG systems, without supplying measurements. Those extra decisions create opportunities to recover from a poor first result; they also introduce additional work.

The control-flow vocabulary comes from Anthropic’s December 2024 article [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents): chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer. Anthropic describes these as workflow patterns and distinguishes prescribed workflows from agents that dynamically direct their own execution. It also describes the cost and latency tradeoffs of adding agentic behavior. Voss uses the five patterns to explain how to structure an agent application.

### Start with a chain, then add routing

[8:47](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=527s)

A **chain** passes the output of one LLM call into the next. In LlamaIndex workflows, regular Python functions perform the work, and type annotations determine how events pass between steps. The event type is the connection: a step emits an event that another step accepts.

A compact Python example can follow the invoice use case through two calls: extract the relevant information, then turn that extraction into a concise summary. The workflow structure below uses the event-based API style described in the talk:

```python
from llama_index.core.workflow import (
    Event, StartEvent, StopEvent, Workflow, step,
)


class InvoiceExtracted(Event):
    details: str


class InvoiceSummary(Workflow):
    def __init__(self, llm, **kwargs):
        super().__init__(**kwargs)
        self.llm = llm

    @step
    async def extract(self, ev: StartEvent) -> InvoiceExtracted:
        response = await self.llm.acomplete(
            "Extract the vendor, invoice date, amount, and payment terms. "
            "Mark missing fields as unknown.\n\n" + ev.text
        )
        return InvoiceExtracted(details=str(response))

    @step
    async def summarize(self, ev: InvoiceExtracted) -> StopEvent:
        response = await self.llm.acomplete(
            "Summarize this invoice in one sentence, preserving unknowns.\n\n"
            + ev.details
        )
        return StopEvent(result=str(response))


async def summarize_invoice(llm, invoice_text: str) -> str:
    workflow = InvoiceSummary(llm=llm, timeout=60)
    return await workflow.run(text=invoice_text)
```

`InvoiceExtracted` carries the first call’s output to the second step. The workflow visualizer shown in the talk makes this sequential structure visible as a chain.

**Routing** adds a model-selected branch. Give the LLM several tools that solve different kinds of problems, or solve the same problem in different ways, and let it choose which path to follow. The chosen branch can contain an entire chain of its own. Voss skips the routing code, but the structural change is clear: the first decision selects a sequence of work rather than forcing every input through the same sequence.

### Run independent work in parallel

[10:21](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=621s)

**Parallelization** runs multiple LLM paths concurrently and aggregates their results. Its first flavor, **sectioning**, applies different operations to the same input. Voss’s example is a guardrail: one track processes the request or drafts an answer while another checks whether the request is illegal or violates the application’s rules. The two questions can be answered independently. If the guardrail rejects the request, its result can prevent the processing track’s answer from being delivered.

![Slide titled “Parallelization: flavor 1 — Sectioning” shows an input branching to LLM Call 1, LLM Call 2, and LLM Call 3, then joining at an Aggregator and flowing to Out.](https://ai.engineer/api/talk-essay-media/8b2008636dedd9bbbf627d4de93f2c9ecd9f3619e4cbb4b732726d37feaa0794)

Sectioning runs three LLM calls in parallel before aggregating their outputs.

The second flavor, **voting**, sends exactly the same query to three tracks. Those tracks can use repeated calls to one nondeterministic LLM, or different models with different capabilities and specialties. An aggregation rule then requires a majority or unanimous agreement.

| Pattern | What the tracks do | How results combine |
|---|---|---|
| Sectioning | Different operations on one input | Combine results or gate delivery |
| Voting | Answer the same question | Require majority or unanimous agreement |

Voss recommends voting to reduce hallucination, reasoning that different calls are unlikely to invent the same answer. Agreement is useful evidence, but it is not independent verification: models can share errors. The later study [Correlated Errors in Large Language Models](https://proceedings.mlr.press/v267/kim25e.html) documents that limitation across models; it does not directly test this three-track example.

Both flavors use the same workflow machinery: **emit multiple events, perform work concurrently, then collect the results**. The difference is in what each worker is asked to do and what the collector decides. A guardrail collector decides whether an answer may be released; a voting collector decides whether the answers satisfy its agreement rule.

### Let an orchestrator determine the subtasks

[12:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=744s)

In **orchestrator-workers**, an LLM looks at a complex task and decides how to split it into simpler tasks. A multipart question becomes several smaller questions, each of which can be assigned to a worker. This moves decomposition into the model’s decision-making process instead of requiring every subtask to be known in advance.

Voss uses deep research to explain the complete path:

1. An orchestrator examines the broad research question and generates related questions to investigate.
2. Workers answer those questions in parallel.
3. A final stage aggregates the worker outputs into one coherent answer.

The synthesis step matters: a set of individual answers is not yet a response to the original question. The workflow must bring them back together. This pattern uses the same parallel execution mechanism introduced for sectioning and voting, but the model determines the work to distribute.

### Evaluate against the original goal, then revise

[13:08](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=788s)

The **evaluator-optimizer** pattern, also called self-reflection, closes a feedback loop around generation. An LLM evaluates whether an output did a good job. To make that judgment meaningful, it receives both the output and the original question—or the original input and the goal. The criterion is whether the requested goal was reached.

If the output falls short, the evaluator returns feedback to the generating step. Voss’s examples are specific defects: the answer hallucinated something, or it missed part of the question. The generator then gets another attempt informed by that feedback. In a workflow, this is a loop: an event sends execution back to the first step instead of proceeding directly to completion.

### Compose workflows and hand control between agents

[13:56](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=836s)

**The patterns become more useful when combined.** A workflow can route a request into a chain, distribute part of that chain across workers, and use an evaluator to send incomplete work back for revision. Voss closes by moving from these control-flow patterns to the tools and agents they coordinate.

A tool begins as a Python function supplied to an agent. Voss describes wrapping it in a step wrapper, but two concepts need to be kept separate: `@step` marks an event-handling workflow method, while an agent tool is a callable the model can select. The current [FunctionAgent guide](https://developers.llamaindex.ai/python/examples/agent/agent_workflow_basic/) passes ordinary annotated functions directly in the `tools` list; a workflow step decorator is not required for those functions.

`FunctionAgent` takes a system prompt, an LLM, and a set of tools. Multiple configured agents can then be supplied to an `AgentWorkflow`, which supports passing control between them. The closing slide shows research, writing, and review agents, a root agent, and initial state fields for research notes, report content, and review. Those fields make the shared work visible: the system has more to coordinate than a sequence of isolated model responses.

![“Multi-agent system” slide with an AgentWorkflow code block listing three agents, a root agent, and initial state fields for research notes, report content, and review.](https://ai.engineer/api/talk-essay-media/85e54b9b5858fc8cb21a59d67b56453def47ffe592cdd8d0a8c99109b5d78d50)

A multi-agent workflow configured with research, writing, and review agents.

Voss calls the multi-agent construction technically one line of code. That describes assembling already configured agents into the workflow, not defining the complete application or all its coordination behavior. The talk ends with a pointer to a fuller notebook tutorial for building a deep-research workflow—the next step after learning the individual patterns and seeing how they fit together.

## Resources

- [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic’s guide to chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer workflows. The live article includes later tooling updates.
- [Bending without breaking: optimal design patterns for effective agents](https://www.llamaindex.ai/blog/bending-without-breaking-optimal-design-patterns-for-effective-agents) — Laurie Voss explains when to constrain agent execution and when to allow model-directed decisions.
- [Introducing LlamaIndex workflows](https://www.llamaindex.ai/blog/introducing-workflows-beta-a-new-way-to-create-complex-ai-applications-with-llamaindex) — The 2024 introduction to typed workflow events, loops, shared context, and visualization, with historical beta code examples.
- [FunctionAgent / AgentWorkflow Basic Introduction](https://developers.llamaindex.ai/python/examples/agent/agent_workflow_basic/) — Current examples for supplying Python tools to an agent, running it, maintaining state, and streaming events.
- [Building a Deep Research AI Multi-Agent with LlamaIndex](https://www.datacamp.com/code-along/building-a-deep-research-ai-multi-agent-with-llama-index) — Voss’s code-along on building and evaluating agents that research news articles.
- [LlamaHub](https://llamahub.ai/) — Registry of LlamaIndex data loaders, agent tools, packs, and datasets.
- [Correlated Errors in Large Language Models](https://proceedings.mlr.press/v267/kim25e.html) — An ICML 2025 study of shared errors across models and their consequences for evaluation and hiring decisions.

## Slides

### Innovation partner and platinum sponsors

AWS is the innovation partner, alongside six platinum sponsors.

[0:10](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=10s) [Sponsor slate naming AWS as innovation partner and six platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/447d7c7ce56420fe6fa5d8a3f155a740cd646365dd5ce27bc5be6f4ace5069bc.webp)

INNOVATION PARTNER
aws

PLATINUM SPONSORS
Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### What are we talking about?

The talk introduces agents and RAG, then surveys five agent design patterns. The speaker says there won't be time to build an agent or multi-agent system.

[0:25](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=25s) [Agenda covering LlamaIndex, agents, RAG, agent patterns and building agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/1ff724c6051e18df1e3347cc93dd323628f3b7de2dc995caf29504d76c2f5819.webp)

What are we talking about?

• What is LlamaIndex
• What is an gent?
• Why build agents?
• What is RAG?
• Why agents need RAG
• Agent Patterns in LlamaIndex
  ▪ Chaining
  ▪ Routing
  ▪ Parallelization
  ▪ Orchestrator-Workers
  ▪ Evaluator-Optimizer
• How to build agents in LlamaIndex

AIE
Microsoft
smol ai

### What is LlamaIndex?

LlamaIndex is a Python and TypeScript framework for building generative AI applications, including agents. The slide points to documentation for each language.

[0:50](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=50s) [LlamaIndex introduction with Python and TypeScript documentation addresses.](https://assets.ai.engineer/sites/aiecode2025/sha256/e88e5f991694132733b04a888e67066e01f50a6e061f71581d3844da1c0ef75d.webp)

What is LlamaIndex?

Python: docs.llamaindex.ai
TypeScript: ts.llamaindex.ai

AIE
Microsoft
smol ai

### LlamaParse

LlamaParse parses PDFs, Word documents, and PowerPoints, making unstructured data easier for an LLM to understand. The slide advertises 10,000 free pages per month.

[1:05](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=65s) [LlamaParse slide describing document parsing and a free monthly page allowance.](https://assets.ai.engineer/sites/aiecode2025/sha256/5f0438b5dda29fd0a6b6c4a1514c7dde8f1fcbdf0419c01bffb2ea65464fa0ad.webp)

LlamaParse
World's best parser of complex documents

Free for 10000 pages/month!
cloud.llamaindex.ai

Event branding: AIE, Microsoft, smol ai

### LlamaCloud

LlamaCloud turns documents into a retrieval endpoint. The paid enterprise service is available as SaaS or as a private cloud deployment.

[1:33](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=93s) [LlamaCloud is presented as an enterprise RAG API available as SaaS or a private cloud deployment.](https://assets.ai.engineer/sites/aiecode2025/sha256/8dfffa664ac99c4ff6e1c6fb047e74727cda5aa88dc05f486472516023d95fe4.webp)

LlamaCloud

Turn-key RAG API for Enterprises
Available as SaaS or private cloud deployment
Sign up at cloud.llamaindex.ai

AIE
Microsoft
smol ai

### LlamaHub

LlamaHub is a registry of open-source integrations for LlamaIndex, including adapters for data sources and vector stores, plus pre-built agent tools.

[1:51](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=111s) [LlamaHub lists data loaders, LLMs, embedding models, agent tools, vector stores and pre-built strategies.](https://assets.ai.engineer/sites/aiecode2025/sha256/b54664380d1ac33463e4e60cd06238fff1437ae582ccc2e0611b5a5eb1fc0c22.webp)

LlamaHub
llamahub.ai

Data Loaders
LLMs
Embedding Models
Agent Tools
Vector Stores
Pre-built strategies

Interface text:
AIE
Microsoft
smol ai

### Why LlamaIndex?

LlamaIndex aims to speed the path from prototype to production by reducing boilerplate and providing best practices.

[2:25](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=145s) [Reasons listed include building faster, skipping boilerplate, avoiding early pitfalls, getting best practices and moving from prototype to production.](https://assets.ai.engineer/sites/aiecode2025/sha256/4478f37332c1e9a7c4e14bdf8b2333bad70835ca601a03061dc5502ddf4d8d33.webp)

Why LlamaIndex?

• Build faster
• Skip the boilerplate
• Avoid early pitfalls
• Get best practices for free
• Go from prototype to production

AIE
Microsoft
smol ai

### What can you build in LlamaIndex?

The speaker highlights AI agents and retrieval-augmented generation (RAG) as two areas where LlamaIndex is particularly strong.

[2:56](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=176s) [Slide lists AI agents and RAG as things to build with LlamaIndex.](https://assets.ai.engineer/sites/aiecode2025/sha256/40779f3603f30439a99759bdc907354930ae160e984797f16ec772b26860f8a0.webp)

What can you build in LlamaIndex?

• Lots of stuff! Especially...
• AI agents
• RAG

AIE
Microsoft
smol ai

### What is an agent?

An agent is semi-autonomous software that uses tools to retrieve information or take action toward a goal without every step being specified in advance.

[3:17](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=197s) [Slide defines an agent as semi-autonomous software that uses tools to accomplish a goal.](https://assets.ai.engineer/sites/aiecode2025/sha256/cb87b4d58a3ce29dcc9e992cd217facdb7ec202cc961904dfed8a372b5d7b481.webp)

What is an agent?

Semi-autonomous software that uses tools to accomplish a goal

AIE
Microsoft
smol ai

### Agentic programming is a new paradigm

LLMs decide which tools to use, giving agents flexibility to handle unexpected or unknown situations.

[3:39](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=219s) [Slide states that agentic programming is a new paradigm.](https://assets.ai.engineer/sites/aiecode2025/sha256/62d368858fb1d4e28b47e66fd20e024a39576c29f58e93b0bcd63baa10724bd1.webp)

Agentic programming is a new paradigm

AIE
Microsoft
smol ai

### When does an agent make sense?

Agents make sense when unstructured data brings messy inputs: their flexibility helps them deal with the unexpected.

[3:53](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=233s) [Slide says agents make sense when data is messy, which is most of the time.](https://assets.ai.engineer/sites/aiecode2025/sha256/785e2ede692943423dc6b1beaa7bce6da93fa9942001cbc71a3eee962aab6e44.webp)

When does an agent make sense?

When your data is messy, which is most of the time

AIE
Microsoft
smol ai

### LLMs turn lots of text into less text

The speaker sees a good agent use case in reducing a large body of text to a smaller one, such as interpreting contracts or summarizing documents.

[4:17](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=257s) [Slide says LLMs are good at turning lots of text into less text.](https://assets.ai.engineer/sites/aiecode2025/sha256/4f638c46324a7fba49f1874c095d7ed169069773bc76f2b9cb70ea3b0dad9a4e.webp)

LLMs are good at turning lots of text into less text

Interface text:
AIE
Microsoft
smol ai

### LLMs work well under the hood

The speaker recommends integrating LLMs into existing software to turn unstructured inputs into structured data for decisions and downstream use.

[4:53](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=293s) [Slide says LLMs work well under the hood and that you do not have to build a chatbot.](https://assets.ai.engineer/sites/aiecode2025/sha256/c2d65c1c956f7789ffd078a4805fb106174c56e056d9ef2089ba0602132b8f6b.webp)

LLMs work well under the hood

You don't have to build a chatbot

Interface text:
AIE
Microsoft
smol ai

### LLMs need data

For company or domain problems, the speaker recommends giving LLMs relevant contextual data and presents RAG as a way to use large amounts of data.

[5:30](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=330s) [Slide reads: LLMs need data. The solution is RAG.](https://assets.ai.engineer/sites/aiecode2025/sha256/8637a112e83225d8747c6f3da7a5885a33e909b1f885f83320b5c808aa800359.webp)

LLMs need data
The solution is RAG

Interface text:
AIE
Microsoft
smol ai

### Agents need RAG

Agents can use RAG to retrieve specific context. An agent can also break a complex question into simpler ones, check its answers, and retry when needed.

[7:07](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=427s) [Slide reads: Agents need RAG, and RAG needs agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/efcfd2daa69ad61f0129e6144d4dd29dab0062b82a5f7ad27cade3623591845c.webp)

Agents need RAG
and RAG needs agents

Other visible text:
AIE
RAG
aws

### Building Effective Agents

The speaker introduces five agent design patterns from Anthropic’s Building Effective Agents, which he recognized from his own work.

[8:17](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=497s) [Overview of five agent patterns: chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer.](https://assets.ai.engineer/sites/aiecode2025/sha256/274f18641d30f6aa5635447ae9770315b9243480f0be498cc8411c91aed26954.webp)

Building Effective Agents

anthropic.com/research/building-effective-agents

• Chaining
• Routing
• Parallelization
• Orchestrator-Workers
• Evaluator-Optimizer

Interface text:
AIE
Microsoft
smol ai

### Chaining

A chain passes each LLM call’s output to the next. After the first call, a gate sends passing output onward and failed output to an exit.

[8:47](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=527s) [A chain of three LLM calls with a gate after the first call; failed inputs exit.](https://assets.ai.engineer/sites/aiecode2025/sha256/afcbcebd34a906b1caff44613eb17f70cd0f57adad9cac5df4d8f9bce83828a4.webp)

Chaining

In → LLM Call 1 → Output 1 → Gate
Pass → LLM Call 2 → Output 2 → LLM Call 3 → Out
Fail → Exit

Interface text: AIE; Microsoft; smol ai

### Workflows

In this LlamaIndex workflow, type annotations connect three Python steps: each receives an event and returns the event consumed by the next step, ending with a StopEvent.

[8:56](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=536s) [Python workflow example with three event-driven steps leading to a stop event.](https://assets.ai.engineer/sites/aiecode2025/sha256/23be0553a1bf03ca84faf7347dd88005857270bf2ccfd9b9c60493962f1bf6dd.webp)

Workflows

class MyWorkflow(Workflow):
    @step
    async def step_one(self, ev: StartEvent) -> FirstEvent:
        print(ev.first_input)
        return FirstEvent(first_output="First step complete.")

    @step
    async def step_two(self, ev: FirstEvent) -> SecondEvent:
        print(ev.first_output)
        return SecondEvent(second_output="Second step complete.")

    @step
    async def step_three(self, ev: SecondEvent) -> StopEvent:
        print(ev.second_output)
        return StopEvent(result="Workflow complete.")

Interface text:
COPY
AIE
Microsoft
smol ai

### Workflow visualization

LlamaIndex’s built-in visualizer shows a chain of three steps connected by events, from StartEvent through StopEvent to completion.

[9:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=564s) [Graph linking a start event through three steps and intermediate events to a stop event.](https://assets.ai.engineer/sites/aiecode2025/sha256/fcdeb7e3a12c9771bf27677553ea7f964e0728fce112af9792eb1b49d14bca35.webp)

Workflow visualization

StartEvent → step_one → FirstEvent → step_two → SecondEvent → step_three → StopEvent → _done

AIE
Microsoft
smol ai

[9:31](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=571s) [A chain of events and steps runs from StartEvent to StopEvent and completion.](https://assets.ai.engineer/sites/aiecode2025/sha256/35906efa62cbb0f616043b4d8ba1e546c2f86ec6bf3a454da001d7eacff242f3.webp)

Workflow visualization

StartEvent → step_one → FirstEvent → step_two → SecondEvent → step_three → StopEvent → _done

AIE
Microsoft
smol ai

### Routing

An LLM router chooses among several LLM-based tools that address different problems or solve a problem in different ways, then returns the selected path’s output.

[9:39](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=579s) [An LLM router directs input to one of three LLM calls, which lead to output.](https://assets.ai.engineer/sites/aiecode2025/sha256/58e8062259e251d4bfbbed0d2ee3e3ad703e8b5441c68281dc3ef0f104c7145e.webp)

Routing

In
LLM Call Router
LLM Call 1
LLM Call 2
LLM Call 3
Out

AIE
Microsoft
smol ai

## Full transcript

[0:00](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=0s) [on hold music] Uh, hi, everybody.

[0:16](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=16s) Uh, you probably met me this morning, uh, when I greeted you all to the conference. My name is Laurie. I'm VP of Dev Relations at LlamaIndex. Uh, today I'm gonna be talking about LlamaIndex and what it is, uh, very briefly because I've only got 15 minutes.

[0:29](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=29s) Uh, and then we're gonna talk about agents and how they are built, plus a very, very brief refresher on RAG, uh, and why it's necessary for agents. Then we're gonna look at some high-level, uh, design patterns that improve the performance of your agent, and we won't have time to build-- to dive into how to build an n-

[0:46](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=46s) an agent as well as a multi-agent system because we've only got 15 minutes. So let's go fast. So what is LlamaIndex? We are a bunch of things. Start with the most obvious.

[0:54](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=54s) We are a framework in Python and TypeScript for, uh, building generative AI applications. We are particularly good at building agents. We also have a service called LlamaParse. LlamaParse, uh, will parse complicated document formats for you.

[1:07](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=67s) So PDFs, Word, PowerPoints, those sorts of things. These are crucial to, uh, building an effective agent, uh, is being able to parse your unstructured data in an effective way.

[1:18](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=78s) And LlamaParse will ev- uh, demonstrably improve the quality of the agents you build by making the data easier for an LLM to understand than if you just try and, uh, feed these things in through an open source parser or something like that.

[1:31](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=91s) Uh, we also have an enterprise service called LlamaParse. Sorry, LlamaCloud. Uh, if what you wanna do is stuff documents into one end and get a retrieval endpoint out of the other, uh, then that is the service for you.

[1:43](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=103s) Uh, unlike the rest of LlamaIndex, it costs money. It is available as a SaaS at cloud.llamaindex.ai, or you can get it deployed, uh, onto your own private cloud. Uh, we also have a website called LlamaHub, uh, which is a huge registry of open source software, uh, that plugs into the framework that integrates with everything.

[2:01](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=121s) So if you need to get your data out of Notion or out of Slack or out of any database in the world, that is where you find the adapters.

[2:08](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=128s) If you wanna store your data in any vector database that exists, the adapters exist for that. And we also integrate with every LLM that exists. So, uh, 400 different models over 80 different LLM providers, uh, including local ones like Llama 3.

[2:23](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=143s) Um, oh, and it also has pre-built agent tools. So if you are building an agent, you can just plug in an existing agent tool without having to build one yourself.

[2:32](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=152s) Uh, why should you use LlamaIndex? Because we will help you go faster. That is the base promise of a framework generally. Uh, you have actual business and technology problems to solve, and you have limited time.

[2:44](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=164s) A framework is gonna help you, uh, get past those with, uh, by skipping the boilerplate, getting best practices for free, uh, and getting to production faster in general.

[2:55](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=175s) Uh, what can you build in LlamaIndex? Well, anything obviously, but there's two things that we are particularly good at. One is Retrieval-augmented generation, uh, and the other one is agents.

[3:04](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=184s) Uh, both of which you are, uh, probably familiar with already from, uh, just being at this conference because we won't shut up about agents at this conference. Uh, it's worth covering why you would want to build those things, though.

[3:18](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=198s) Uh, so what is an agent? An agent is, uh, a dramatically overused term in the industry. Just about everything is an agent in 2025. Uh, but what I mean when I say an agent is it is a bit of semi-autonomous software that can use tools to achieve a goal without you having to explicitly specify what steps it's

[3:35](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=215s) gonna take to achieve that goal. Uh, and the tools can do anything, which is great. They can take-- retrieve information, or they can take action. Uh, agents are a dramatic departure from traditional programming.

[3:47](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=227s) Uh, LLMs are given decision-making power to decide what tools to use and stuff like that. That makes them extremely flexible and powerful. So the time to build an agent is when that flexibility, that, uh, ability to deal with the unexpected or the unknown, uh, is gonna come in useful.

[4:04](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=244s) And when that's really useful is when you have a bu- a bunch of unstructured data. Uh, LLMs are extremely good at handling messy inputs, and the world is full of messy inputs, so there's a whole lot of applications for LLMs.

[4:16](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=256s) Um, in general, I regard a good agent use case as any situation where an LLM is required to turn a large body of text into a smaller body of text.

[4:25](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=265s) I think that is a key principle of agent design and LLM use in general, is that they are not good at taking a small prompt and turning it into a big body of text.

[4:33](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=273s) They are very good at summarizing stuff down. So interpreting a contract, processing an invoice, uh, applying regulations, summarizing documents, uh, a thing where you need to turn text into less text.

[4:45](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=285s) So a calendar event, a decision, a report, an answer to a question, those are good applications for LLMs and good applications for agents. The most obvious application of LLMs is a chat interface where you give it questions and it answers.

[4:58](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=298s) I encourage you to think beyond the chatbot. Uh, chatbots are a very 2023 way, way of using an LLM. Uh, we believe a much greater addressable surface is if you integrate them into existing software.

[5:12](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=312s) So you use the LLM's capability to handle messy inputs, to handle unstructured data, and turn it into structured data that it can then fee-- make decisions about and feed into your regular software.

[5:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=324s) That is a really productive and powerful set of use cases, uh, and we think it's much bigger than the market for chatbots. Um, I've talked a lot about unstructured data.

[5:36](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=336s) The reason I talk about it is because that is where LLMs become really useful. Um, unless you are building something extremely generic, uh, you are not going to be able to get anything useful out of an LLM by just asking it questions.

[5:49](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=349s) You're going to have to give it contextual data relevant to your company, to your domain, to whatever problem set it is that you are working on. Uh, so you have to feed the LLM your data, and the problem is that you have tons of data, and this is the use case for RAG.

[6:03](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=363s) Uh-

[6:05](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=365s) You take all of your data, you embed it, which is turning it into vectors that you can search for in a vector search, uh, and the-- you can then take your query, take your questions about your data, uh, and turn-- embed them in this-- into the same vector database, and they will end up mathematically nearby in vector

[6:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=384s) space to the context that you fed in that is relevant. So instead of having to take all of your data and feed it to the LLM every single time, which would be tremendously slow and tremendously expensive, you can just feed in the most relevant context out of your data corpus, uh, and, uh, answer questions about that.

[6:43](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=403s) Um, this is why RAG will never die. People keep talking about like larger and larger contexts and, you know, more and more powerful models. Uh, it's always going to be cheaper and faster to send less data that the LLM has to think about less.

[6:56](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=416s) It's always going to be, uh-- You're always going to get at more answers, better answers if the context that you have given your LLM is, uh, more specific. Uh, so agents can use RAG as one of their tools, but also, uh,

[7:12](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=432s) um, agents-- So agents need RAG, uh, but RAG also needs agents. RAG by itself, naive top K RAG, where you just, you know, throw in a query and retrieve the, uh, retrieve the most relevant context and feed that to the LLM, that's not going to work very well for a variety of situations.

[7:29](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=449s) But what we've found through lots of production use cases is that layering an agent on top of your RAG, uh, will produce significantly higher quality results, um, and they're capable of doing things that RAG just can't do.

[7:44](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=464s) RAG is a simple question and answer, uh, robot, whereas an agent can do stuff like introspection. Like, could this complicated question be answered more easily if it were a series of simpler questions?

[7:55](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=475s) Uh, do I need to try extracting that data again because the data that I got out is nonsense? Did I just give a sensible answer to th- your question, or should I try again?

[8:03](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=483s) That is something that an agent can do. It can look at its own responses and improve itself, which RAG by itself obviously doesn't do. Uh, so agents improve the performance of RAG, both in terms of speed, uh, and crucially in accuracy.

[8:18](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=498s) Um, in December of last year, Anthropic did an excellent post about how to build agents, uh, in which they codified, uh, some design patterns about how to build an effective agent that we immediately recognized, uh, from our own work, uh, building agents.

[8:33](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=513s) So I'm going to go through them very quickly, and given the amount of time I have, that's probably all I'm gonna be able to cover. Um, they are chaining, routing, parallelization, orchestrator workers, and evaluator optimizers.

[8:47](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=527s) Um, the first and most obvious is the chain. Uh, you can use an LLM to do some work, and you pass the output of that to another LLM, and you pass the output of that to another LLM.

[8:55](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=535s) Uh, it is trivial to build, uh, especially in LlamaIndex. Uh, this is what, uh, a chain looks like in LlamaIndex. We use, uh, an abstraction called workflows, uh, where you define regular Python functions that do whatever you need them to do, uh, and you use event annotations, uh, to define how you-- Sorry, you use type annotations to

[9:15](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=555s) define how events pass from step to step within a workflow. Uh, it is a very simple and flexible pattern, uh, that our users like a lot. Um,

[9:26](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=566s) so LlamaIndex workflows have a built-in visualizer, the output of which you can see here. Uh, this is obviously a chain. Um,

[9:34](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=574s) there is much, much more to LLM applications than a chain though, despite, uh, what the names of some other frameworks might indicate. Um, the next pattern Anthropic called out is routing.

[9:44](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=584s) Uh, in this one, you create several LLM-based tools, uh, to solve a problem, uh, in different ways or to solve different types of problems, and you give the LLM decision-making power to say which of these tools should I call?

[9:57](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=597s) Which of these different LLM paths should I follow? Uh, again, not that complicated a con-- uh, concept and simple to build, uh, in LlamaIndex. I'm gonna sh-- uh, spare you the code this time.

[10:07](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=607s) You can do it using w-- uh, branches. You can just decide that you're going to split off into your own chain, uh, and do a series of, uh, do another series of work, uh, based on the or-original decision made by the LLM.

[10:21](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=621s) Um, the next pattern is parallelization, which is where things begin to get interesting. Uh, Anthropic defines this as running several LLMs in parallel and then aggregating their results. Uh, and they define parallelization as having two flavors.

[10:34](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=634s) Uh, the first is sectioning. Um, this is where you take the same input, and you act on it in completely different ways. The, uh, sort of canonical use case of this is guardrails.

[10:44](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=644s) So the user has a query or a piece of input that they want processed, uh, and you use one of your tracks to actually process the data or to answer the query, and you use the second one of your tracks to query, is this an illegal request?

[10:58](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=658s) Is this against my rules? Is this-- Those two ques- those two questions are related, but they can be answered in parallel, and you can use your guardrails to cut off the answer, uh, from your processing if it turns out to be, uh, you know, illegal or otherwise undesirable.

[11:13](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=673s) Um, the other par-- flavor lo-- flavor of parallelization is voting. This is where you take exactly the same query, and you give it to, uh, three different, uh, three different tracks.

[11:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=684s) The tracks can be literally exactly the same LLM because they are non-deterministic, so it might not give the same answer every time. Or you could give it to multiple different LLMs which have different capabilities and different specialties, and then you take the answers, and you allow them-- and you see if, uh, the tracks came to the same

[11:39](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=699s) answer. You can take a majority vote. You can take a unanimous vote. And what this does is it allows you to, uh, limit the amount of, uh, uh, hallucination that is happening.

[11:51](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=711s) Uh, it's a great way of reducing hallucination because if, you know, three different LLMs come to the same conclusion, then it's probably not the LLM just making stuff up because LLMs hallucinate, but they hallucinate in different ways, so they seldom ha-hallucinate to the same answer.

[12:06](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=726s) Uh, whichever flavor you use, it's implemented the same way, uh, using concurrency. Um, we-- uh, uh, L- LlamaIndex workflows allow you to emit multiple events simultaneously, uh, and then collect those events, uh, at the other end, so you can do s- you can do work concurrently.

[12:24](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=744s) Uh, and, uh, yeah. And it-- I think the visualization here is particularly pretty. Um, the next pattern is orchestrator workers. Uh, you can use an LLM to look at a complex task, like a multi-part question, uh, and split it into several simpler questions and ask each of those questions in parallel.

[12:43](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=763s) So this is how deep research works, basically. Uh, it takes a very complicated question and it says, "I'm going to, I'm going to look at all of the possible questions that could come from this deep question, and I'm going to answer them all at the same time, and then I'm going to aggregate all of the answers that

[12:57](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=777s) I've got and turn them into one single coherent answer." This is a very powerful pattern that is, uh, doing a lot of good in the world right now. Uh,

[13:08](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=788s) this is also implemented using parallelization. Um, and the final pattern that Anthropic called out is the evaluator optimizer, which is also called self-reflection. Uh, in this pattern, you use the LLM to decide whether or not the LLM has done a good job.

[13:22](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=802s) So, uh, you take your output, you feed it to an LLM, and you say, "Here was the original question," or, "Here was the original input and the goal that I had.

[13:30](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=810s) Have you actually reached the goal that I had?" Uh, and if not, you can get the LLM to generate feedback and send it back, uh, to the original first step and say, "Okay, you almost got the answer, but you hallucinated something," or, uh, "You missed a part of the question," or, you know, something like that.

[13:46](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=826s) Um, this is again easy to do in LlamaIndex. In workflows, you just create a loop, uh, and you can send yourself back to this step one. Um,

[13:56](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=836s) and the real power here is obviously combining all of these patterns. You can create arbitrarily complex workflows, uh, to handle any combination of circumstances. Um, right near the beginning, uh, when I was defining an agent-- I have sixty seconds left, so I'm just gonna go through to the syntax super quick.

[14:12](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=852s) Uh, I said it was-- that agents are defined by their ability to use tools. In LlamaIndex, this is what a tool definition looks like. It is just a Python function that you have wrapped, uh, in a step wrapper.

[14:23](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=863s) Um, and the way that you use your tool function is you just give it to an agent, and the agent will figure out that it is a tool function and start using it.

[14:33](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=873s) Uh, this allows you to create workflows that are multi-agent systems. I do not have time to, uh, explain how multi-agent systems work for, uh, this, but this is how you create a multi-agent system in, in LlamaIndex.

[14:47](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=887s) Uh, you create, uh, a FunctionAgent which gets-- takes a system prompt, it takes an LLM, uh, and it takes, uh, a set of tools, and you can feed, uh, an array of agents into a multi-agent system, which then just sort of figures it out by itself, passing control back from one agent to another.

[15:05](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=905s) Uh, this is technically one line of code, and we're pretty proud of it. Uh,

[15:10](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=910s) and that is about it. Um, if you want a full agent workflow and workflows tutorial, this was the simplest possible one. The-- It is available, uh, at that. This little-- this notebook will teach you how to build a deep research of your own.

[15:23](https://www.youtube.com/watch?v=72XxWkd8Jrk&t=923s) Uh, and with that, I am pretty much out of time. If I-- if you have any questions, I will be outside in the hallway. Thank you very much. [clapping] [outro jingle]
