AI Engineer World's Fair 2026

Your Agent Evolved. Your Evals Didn't.

Ameya Bhatawdekar· VP, Field CTO, Braintrust24:13

Loading slides…

Read the talk

Your Agent Evolved. Your Evals Didn’t.

As agents move from single model calls to tool loops, memory, and code execution, evaluations must expand from answer quality to the behavior of the whole system.

From a talk by Ameya Bhatawdekar

Before you start: Familiarity with model prompts, tool calls, and basic retrieval-augmented generation will help; no evaluation framework experience is required.

Why a better model can require a different system

Why is an AI demo easy to build, while maintaining its quality in production is so difficult? A chatbot, agent, or batch-processing application can change substantially after its first working version: prompts become pipelines, orchestration grows, and eventually the application needs a rewrite. Ameya Bhatawdekar, Braintrust’s Field CTO, frames this as a moving-target problem. Models change, users discover different ways to use the application, and the underlying data changes too.

Slide titled “The core question” asking why building, shipping, and maintaining AI agent quality is hard, with the speaker inset at left.Open full source frame
The core question: why is it so hard to build, ship, and maintain the quality of AI agents?

Better tool calling, longer usable context, code generation for sandboxed execution, and practical memory systems can each change what an application is capable of doing. These are sometimes large enough advances to require re-platforming, rather than another round of prompt tuning.

Simply replacing the model may leave those capabilities inaccessible. If the old model struggled to choose tools, the surrounding software probably made those choices for it. A stronger model placed inside the same constraints still cannot exercise its new judgment. Restructuring the application can unlock that capability, but it also introduces places where behavior can go wrong. Architecture follows model capabilities; evaluations must follow architecture.

0:000:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Start with an SRE agent—and the simplest evaluation

Consider an SRE agent that can read system information and take operational actions. Its tools include deployment rollback, escalation to a human, and paging. The distinction between read tools and write tools matters: the system can change the environment, not merely describe it. Bhatawdekar uses this agent to ground a series of notional evaluations as the architecture evolves.

The starting architecture is one input, one model call, and one output. Here, the final answer is a reasonable unit of evaluation. Does it answer accurately? Are its claims factual? Does it invent information or rely on stale training knowledge rather than the information relevant to the task?

A golden dataset supplies representative inputs and expected behavior. Scorers encode what a good answer looks like and evaluate the model’s responses against that definition. This is a useful baseline precisely because the architecture is narrow: there is no retrieval stage, tool execution, or orchestration to inspect independently.

4:485:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:48 · section reference included

A chain creates intermediate failure points

Retrieval-augmented generation adds work before the answer. A typical chain follows a fixed procedure:

  1. Parse the user’s input to extract information needed for retrieval.
  2. Retrieve relevant information.
  3. Assemble that information into context.
  4. Ask the model to reason over the context and synthesize an answer.

The answer is now the product of several stages, each of which can affect correctness.

StageFailure to evaluate
ParsingExtracting the wrong information
RetrievalReturning the wrong context
Context constructionOverloading the model with material it cannot use effectively
GenerationFailing to reason correctly over the supplied context

A larger advertised context window does not itself establish that a model can reason well over everything placed inside it. Bhatawdekar recalls early systems in which context stuffing impaired performance even as window sizes increased. Intermediate evaluations make a bad final answer actionable: they help distinguish a retrieval problem from a reasoning problem.

6:567:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:56 · section reference included

Letting the model choose the next step

A fixed chain performs the same kind of sequence for every request. ReAct offers a different structure: interleave reasoning and action, allowing the model to decide what to do next. The paper appeared in 2022; Bhatawdekar places its growing popularity among application builders in 2023 and early 2024.

The loop is straightforward: choose a tool call, inspect its return, reason about the new information, and select the next step. Continue until the user’s intent is satisfied or the iteration budget is exhausted. Instead of requiring a developer to predefine the entire route, the model can organize the work around the request and the observations it receives.

The early models made that flexibility difficult to use reliably. They supplied incorrect tool arguments, selected the wrong tools, made reasoning mistakes, and struggled to preserve useful context over longer executions. An appealing orchestration pattern still depended on capabilities the model did not consistently possess.

8:058:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:05 · section reference included

Workflow graphs buy predictability with explicit control

When the model could not reliably control execution, teams moved control into software. A graph or state machine encoded planning, orchestration, and execution; models performed narrower tasks inside individual nodes. This made behavior more predictable because the surrounding program constrained how the system could proceed.

Those constraints also defined the system’s supported distribution of requests. An intent that fit the graph could work well. An intent requiring a different sequence could expose a gap in the orchestration. Teams responded by adding branches and special cases, increasing the graph’s complexity as they expanded its coverage.

Evaluation consequently expanded beyond the answer and the individual model call:

  • Branch behavior: Does the system choose the appropriate path and behave consistently along it?
  • Inter-node contracts: Does one node produce information the next node can correctly consume?
  • Classifier accuracy: Does an early classification send the request down the right workflow?
  • Node behavior: Does each bounded model task work on its intended inputs?
  • Retries and orchestration: Do retry loops and the overall execution behave correctly?

These checks supplement earlier evaluations. Moving decisions into deterministic software does not remove the need to evaluate those decisions or their interaction with model outputs.

9:469:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:46 · section reference included

The loop returns, and evaluation becomes a distribution

Bhatawdekar places widespread production use of graph frameworks in late 2024 and early 2025. In his account, Anthropic and OpenAI releases in mid-to-late 2025 then substantially improved tool calling, orchestration, long-horizon planning, and the ability to recognize mistakes and course-correct. This is an architectural assessment rather than a model-specific benchmark: he identifies no model versions or measured success threshold.

Graphs built to compensate for earlier weaknesses could still constrain those stronger models and retain their own brittleness. The model-driven loop became practical again. Yet successful execution did not imply a fixed path: repeated runs on the same input could take substantially different trajectories while arriving at the correct answer.

A single successful trace is insufficient evidence of repeatable behavior. Run the same evaluation multiple times and inspect the distribution of results. Bhatawdekar proposes repeated trials without specifying a fixed value of k or an uncertainty calculation; the required number depends on the reliability question being investigated.

MetricSuccess condition across k attemptsQuestion
pass@kAt least one attempt succeedsCan the agent solve the task?
pass^kEvery attempt succeedsCan it solve the task consistently?

The talk’s verbal description of pass^k as how many trials succeed needs a precise distinction. In the original τ-bench definition, it is the probability that all k independent, identically distributed trials succeed, averaged across tasks—not the count or fraction of successful trials. That definition dates to 2024; these metrics become useful here because the architecture makes repeated execution especially consequential.

For a single task with independent trials and success probability p, the distinction is:

pass@⁡k=1−(1−p)kpass⁡k=pk\begin{aligned} \operatorname{pass@}k &= 1-(1-p)^k \\ \operatorname{pass}^{k} &= p^k \end{aligned}

More attempts increase the chance of seeing at least one success, while requiring every attempt to succeed becomes more demanding. A high pass@k therefore does not establish dependable repeated performance.

Slide titled “Gen 5: the modern loop returns,” showing an Input, LLM, Tools, and Answer diagram, an evaluation example, failure modes, and multi-trial metrics including pass@k and pass^k.Open full source frame
The modern tool loop: evaluate a distribution over traces using repeated runs and reliability metrics.

A small TypeScript helper can keep those questions separate when inspecting one batch of graded SRE-agent runs. Each boolean represents whether a run met the evaluation’s success criteria; the helper summarizes the batch, rather than estimating a population probability from it.

typescript

type BatchResult = {
  attempts: number;
  successes: number;
  successFraction: number;
  anySucceeded: boolean;
  allSucceeded: boolean;
};

function summarizeTrials(passed: readonly boolean[]): BatchResult {
  if (passed.length === 0) {
    throw new Error("At least one graded trial is required");
  }

  const successes = passed.filter(Boolean).length;
  return {
    attempts: passed.length,
    successes,
    successFraction: successes / passed.length,
    anySucceeded: successes > 0,
    allSucceeded: successes === passed.length,
  };
}

Keeping the success fraction, any-success condition, and all-success condition distinct prevents a capability result from being mistaken for a reliability result. The traces behind those outcomes then help locate the sources of failure.

12:5413:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:54 · section reference included

The agent becomes a product system

The next expansion surrounds the loop with persistent capabilities. Memory systems store and retrieve information within a session and across sessions, allowing later runs to benefit from earlier ones. Sandboxes provide an execution environment for model-generated code. The behavior being evaluated now depends on more than the current prompt and the model’s immediate tool choices.

Model Context Protocol connections and skill directories add extensibility. Skills augment the agent through symbolic instructions, expanding what it can do without changing model weights. Memory, execution environments, and these extensions all add surfaces where the product can fail. An evaluation suite inherited unchanged from the previous generation covers only part of the resulting system.

16:0616:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:06 · section reference included

Production must keep changing the evaluation suite

Across these architectural generations, evaluations remain a durable description of intended behavior. Their implementation and coverage must change, but they preserve a useful contract: new capabilities should not break the experiences that already work.

That contract needs a continuous connection to actual usage. Production data supplies cases for evaluations; those evaluations guide improvements; the improved system generates new production evidence. Bhatawdekar reports that many teams endorse this flywheel but leave their suites largely static. Even without an architectural change, a suite can become less representative as users and their requests evolve.

The harder requirement is discovering failures the existing suite does not know how to look for. Collecting another example of a known error improves coverage within an established category. Discovering a new category changes the definition of what must be evaluated. Architectural changes make this second kind of discovery especially important: a new component or execution pattern can fail in ways the previous definitions of good behavior never anticipated.

17:4918:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:49 · section reference included

Finding missing categories with Topics

Braintrust combines evaluations, observability, and production insights to feed new cases back to the development team. Its Topics feature adds clustering as a way to find patterns in production data, including failure categories a team has not already encoded in a scorer or guardrail.

Bhatawdekar describes this as cluster analysis over all production data. The operational scope is more specific: Braintrust’s later engineering account describes clustering sampled, extracted facets and then classifying incoming data. The mechanism supports discovering patterns; it does not guarantee discovery of every failure. The useful result is a surfaced category that prompts the team to investigate behavior its existing checks missed.

Those observations can become evaluation cases, expanding the suite beyond new examples of already-known failures. After an architectural change, this gives teams a route from unexpected production behavior to a dataset that explicitly exercises the new failure mode.

20:2620:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

20:26 · section reference included

Make evaluation part of how the system changes

Bhatawdekar expects further model capability unlocks and, with them, more substantial changes to agent architectures. The operational requirement is to make those changes without losing existing capabilities: the system should improve at new tasks while continuing to perform the work users already depend on.

That requires evaluation discipline supported by tools, automation, and an everyday production-feedback workflow. The same process must accommodate both small improvements and changes large enough to reshape the system. Evaluations earn their durability by continuing to describe—and test—the behavior that matters as the implementation underneath them changes.

Takeaway slide stating that models and architectures will keep changing, evaluations should stay updated, and teams should build the flywheel.Open full source frame
Keep evaluations updated as models and architectures change. Build the flywheel.
22:1122:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

22:11 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…