AI Engineer World's Fair 2025

Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents

Aparna Dhinakaran· Co-founder, Arize14:25

Loading slides…

Read the talk

Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents

Agent evaluation starts with individual tool calls, expands to trajectories and conversations, and closes the loop by checking whether the evaluators themselves are right.

From a talk by Aparna Dhinakaran

Before you start: You should be familiar with an LLM agent calling tools and using conversation context to decide what to do next.

When a spreadsheet cannot tell you what improved

A team swaps prompts in an Excel sheet, compares a few model outputs, and ships the version that feels better. Meanwhile, the model and tool definitions may also be changing. Which change actually helped, and on which examples? Without a systematic comparison, even a promising result is difficult to turn into a repeatable development process.

The problem also involves who can participate. Product managers and subject-matter experts need a way to contribute judgments to the same evaluation process. Aparna Dhinakaran, an Arize cofounder, frames agent development around making those judgments systematic. Once an agent reaches production, the question becomes more specific: which sub-agent or tool call consistently produces poor responses, and what needs to change there?

Slide titled “Building Agents is Hard!” with three problem boxes beside a spreadsheet of prompts and model outputs.Open full source frame
Building agents is hard: tracking experiments, involving stakeholders, and iterating in spreadsheets.

Evaluation needs to follow the scope of the behavior being tested. A tool call needs checks on its selection and inputs. A trajectory needs checks across the sequence of calls. A conversation needs checks across turns, where earlier context should inform later answers. Improving the agent also requires improving the evaluations that identify its failures.

0:340:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:34 · section reference included

Separate tool selection from argument correctness

An agent with several tools must first decide which action fits the current situation. That decision depends on the user's request, previous conversation context, and actions already taken. After choosing a tool, it must extract the arguments from that context. These are separate opportunities for failure.

CheckQuestion
Tool selectionIs this the appropriate tool for the next action?
Argument correctnessDo its inputs match the request, context, and required parameters?

A correct tool name does not establish that the call is correct. Keeping the two judgments separate makes the result actionable: a routing problem and an input-extraction problem require different fixes.

An agent branches to three tools: lookup_sales, data_analysis, and make_purchase, each with a listed set of arguments.Open full source frame
Tool calling evaluation considers both the tool choice and its arguments.
3:133:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:13 · section reference included

Start with the paths that need attention

The demonstration moves into Arize's own copilot, an agent that helps users investigate application performance, troubleshoot problems, and find possible improvements. Arize records traces of those user questions and evaluates the interactions using its own tooling. Those traces make the steps behind a response available for inspection.

Before opening an individual trace, Dhinakaran looks across the paths the agent can take. The copilot uses an orchestrator-worker pattern: a top-level planner chooses where to send a request, and tools perform the next actions. A tool's output can lead to another router or orchestrator that chooses a subsequent tool. The resulting response may therefore depend on several layers of decisions.

At the planner level, the useful question is where those paths diverge in quality. Dhinakaran describes the displayed copilot search Q&A results as roughly half correct and half incorrect. The recording does not supply a sample size, collection period, model, or evaluator configuration for that approximate result. It does provide a direction for investigation: search deserves attention before paths whose evaluations look healthier.

4:154:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:15 · section reference included

Follow a failed answer down to its arguments

The aggregate view narrows the search; the trace supplies the diagnosis. Dhinakaran follows a concrete drill-down procedure:

  1. Select the search Q&A correctness evaluation.
  2. Filter to examples marked incorrect.
  3. Open a failing trace and inspect its whole-trace evaluations.
  4. Inspect the individual tool calls, separating function selection from argument correctness.

The trace can contain both a failed overall answer and a successful local decision. That distinction is what makes the next inspection useful.

In the selected example, the function-call evaluation is correct, but the arguments are marked incorrect. The evaluator's explanation says the arguments do not align with the required parameters. Dhinakaran identifies the likely problem as supplying the wrong arguments for the conversation context, despite selecting the appropriate tool. The demonstration reaches a diagnosis; it does not show an implemented correction.

A small Python fixture illustrates why the two results should remain separate. Here, a request to search for dashboard duplication selects search, but supplies a query about monitor alerts. Exact comparison is sufficient for this fixture; the real diagnostic question is whether the arguments express the user's intended request.

python

request = "Find documentation on duplicating a dashboard."
expected = {
    "name": "search",
    "arguments": {"query": "duplicate a dashboard"},
}
actual = {
    "name": "search",
    "arguments": {"query": "configure monitor alerts"},
}

checks = {
    "tool_selection_correct": actual["name"] == expected["name"],
    "arguments_correct": actual["arguments"] == expected["arguments"],
}

assert checks == {
    "tool_selection_correct": True,
    "arguments_correct": False,
}

A single failure label would obscure the useful fact that selection succeeded. Separate results preserve that information while directing the investigation toward argument construction.

7:057:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:05 · section reference included

Correct calls can still form a bad trajectory

The copilot's search path invokes multiple sub-tools. Evaluating each call independently therefore leaves another question unanswered: did the agent call them in the right order? Trajectory evaluation examines the sequence across an entire trace, rather than treating every action as an isolated decision.

For repeated instances of a task, Dhinakaran asks whether the agent consistently completes the necessary actions and converges on an appropriate number of steps. A run that veers into a different order can spend more tokens on the same request, or produce an incorrect output because the actions no longer form a valid sequence. The concern is both completion quality and the work required to get there.

The displayed tool-calling-order evaluation is marked incorrect. That adds a distinct failure category to the earlier argument problem: the individual actions may need attention, but so may their arrangement. No target step count or measured token savings are supplied. A trace-level judgment still covers only one interaction, which leads to the next scope of evaluation: the conversation.

8:408:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:40 · section reference included

Carry context across the conversation

An example with three human-agent back-and-forths introduces behavior that a single trace cannot fully describe. Does the agent maintain a consistent tone? Does it ask the same question repeatedly, despite having already received the answer? Repetition can expose a failure to use earlier interactions, even when an individual response appears reasonable on its own.

The core requirement is to use context from the previous N−1 turns when answering turn N. That makes session evaluation broader than checking each answer separately:

  • Tone consistency: Does the agent maintain an appropriate tone across turns?
  • Context retention: Does it use information supplied earlier instead of asking for it again?
  • Conversation-wide correctness: Does it correctly answer the questions throughout the interaction?

Dhinakaran then opens a conversation from another project to inspect both answer correctness and whether earlier context has been missed. Session evaluations complement the tool and trace checks by testing what survives between interactions.

Arize session detail showing human questions about a noisy monitor and duplicating a dashboard, green AI response cards, and evaluation labels on the right.Open full source frame
A session view shows successive human questions, AI responses, and evaluation labels.
10:2510:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:25 · section reference included

Improve the evaluator alongside the agent

Tool, trajectory, and session evaluations all feed the application-improvement loop. Evaluate the agent, identify where it fails, annotate or refine the outputs, and use those examples to improve its prompt. The evaluations determine which cases enter that process, so their quality directly affects what the team chooses to fix.

An evaluator can be the component that is wrong. With an LLM acting as a judge, its prompt should not remain static while the application evolves. Inspect the evaluation outcomes themselves: an application response marked incorrect may have been acceptable, and the judge may have misclassified it. Treating every negative evaluation as an application defect would send the improvement process after the wrong target.

The evaluator needs its own iteration process: identify mistaken judgments, improve the evaluation prompt, and build and continually refine a golden dataset. The two loops operate together:

LoopEvidence to inspectWhat changes
ApplicationFailed outputs and corrected examplesApplication prompt
EvaluatorMistaken judgments and reference labelsEvaluation prompt and golden dataset

The self-improving stack presented here is a workflow of repeated evaluation and refinement, not a demonstrated autonomous optimizer. Its effectiveness depends on maintaining both the agent's behavior and the quality of the judgments used to improve it.

To try these practices in an application, Dhinakaran points to Arize Phoenix, which she describes as completely open source. For present-day reuse, its current Elastic License 2.0 restricts offering substantial functionality as a hosted or managed service. She also points to Arize AX for running evaluations on your own data.

11:4011:50
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:40 · section reference included

Resources

From the talk

  • Source code and setup instructions for tracing agents, evaluating outputs and running dataset-based experiments.

Updates since the talk

  • Current documentation explaining the two products and their deployment and operational differences.

  • An August 2025 tutorial on grouping traces into conversations and evaluating session correctness, frustration and goal achievement.

  • A March 2026 Phoenix tutorial on tool selection, argument correctness and refining application and evaluator prompts together.

Transcript download
Full transcript

Loading transcript…