AI Engineer World's Fair 2026

Your Agent Failed in Prod. Good Luck Reproducing It.

Loading slides…

Read the talk

Your Agent Failed in Prod. Good Luck Reproducing It.

A successful API call can still execute the wrong trade. Recording agent boundaries turns an otherwise elusive failure into a repeatable test of the code that must stop it.

From a talk by Tisha Chawla and Susheem Koul

Before you start: Familiarity with LLM tool calls, function inputs and outputs, and basic automated testing will help you follow the replay example.

The production failure that disappears locally

An agent calls the wrong tool or writes the wrong data in production. You retrieve the prompt from telemetry, send it to the same model, and run the workflow locally. It works. Further reruns work too. The damaging run is the one you cannot recover, so a successful retry tells you little about whether the next customer is protected. For Tisha Chawla and Susheem Koul, whose agents interact with production backends, this is a debugging problem with consequences beyond a bad answer.

Consider their broker scenario. A user asks to sell $1,000 of stock. Instead of converting that dollar amount into shares, the agent copies 1000 into the order’s quantity field. At $190 per share, the illustrated 1,000-share order sells $190,000 rather than the requested $1,000. The tool has received a valid number with the wrong meaning.

In the speaker-presented scenario, the broker returns HTTP 200 in 30 milliseconds, with zero exceptions and zero alerts. Those are scenario details, not an independently measured incident. The green dashboard accurately reports that the API accepted the request; it does not establish that the trade matched the user’s intent. Successful execution and correct action are different properties.

Slide contrasts “asked: sell $1,000” with a circled red “sold: $190,000.” A green broker response shows 200 OK and FILLED.Open full source frame
Asked to sell $1,000; sold $190,000 with a green 200 OK response.
0:000:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Temperature zero does not freeze the serving system

Turning temperature down to zero does not repair the dollars-to-shares mistake. Greedy decoding can consistently choose a wrong answer. Nor does greedy selection alone guarantee identical output across requests: choosing the highest-scoring token is repeatable only when the scores themselves remain the same. The talk cites engineering reports of dozens of different responses across a thousand identical-prompt runs; that observation should not be treated as a universal failure rate.

The numerical explanation separates several mechanisms:

  • Sampling versus system determinism. An argmax rule specifies how to select from the logits, not how the serving system computes those logits.
  • Floating-point arithmetic. Addition is not associative at finite precision. Different reduction orders can produce slightly different scores; if the leading tokens are sufficiently close, the winning token can change.
  • Batch composition. The talk contrasts an isolated matrix operation under fixed conditions with a request grouped alongside changing traffic. Batch-dependent computation can change the numerical path even when the request itself is unchanged.
  • Expert capacity. In a capacity-limited mixture-of-experts implementation, competition for an expert can change how tokens are handled. Overflow and rerouting depend on the implementation; they are not properties of every MoE model.

These mechanisms explain why controlling a request parameter is weaker than controlling the inference stack. The isolated-operation comparison also requires fixed execution conditions, rather than a guarantee across hardware, releases, or kernels.

2:262:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:26 · section reference included

Recover the run instead of regenerating it

For the broker, identical prose is not the engineering objective. The consequential question is whether the system executes an acceptable state transition. Once a bad decision has occurred, debugging needs that particular decision available again—not another attempt to persuade the model to produce it.

PropertyWhat it asks forEngineering role
Bitwise determinismSame input, identical outputControl generation
ReplayabilityReconstruct a recorded runInspect and retest execution

Replayability changes the dependency: capture what the model actually produced, then use that captured result when investigating the failure. The speakers favor leaving room for exploratory generation because they value its creative benefits. That preference does not require every application to use randomness; the essential point is that debugging should not depend on regenerating the same answer through an uncontrolled hosted stack.

4:515:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:51 · section reference included

Record semantic boundaries

A network recorder cannot see every meaningful agent operation. Local retrieval, in-process tools, and memory access may never issue a network request. The useful recording surface is therefore the boundary of each node: what entered the operation and what left it. Those inputs and outputs preserve the meaning of a step more directly than packets do. Streaming and asynchronous execution make preserving that structure especially important.

The workflow is to annotate, record, visualize, understand, fix, replay, and verify. Recording supplies the evidence for diagnosis; replay makes that evidence usable as a test input. If all model boundaries are stubbed with their recorded outputs, the test can run offline without making model calls.

Chronicle is the proof of concept used to demonstrate this approach. A boundary annotation wraps a method, whether it calls an LLM, retrieves RAG context, or executes a tool. The wrapper records the method’s input/output pair, with metadata such as the model version and code version. Together, those records form the trace.

Editor diagram shows a LIVE boundary wrapper around place_order, captured input, function code, captured output, and an envelope stored in fixtures/traces/.Open full source frame
A boundary wrapper captures method inputs and outputs and stores them as a trace.

The recording is an execution envelope, not a snapshot of every external system: side effects inside a boundary are not automatically captured. There is also a version distinction when following the demonstration: current Chronicle documentation uses ReplayPlan for stub/live selection and still lists streaming capture as planned. The demonstrated boundary mechanism should not be read as support for every streaming or asynchronous configuration.

6:146:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:14 · section reference included

Find where dollars became shares

The stock agent has three annotated methods. A planning LLM receives the user’s request, the place_order tool performs the trade, and a finalization LLM prepares the response. Recording each method gives the trace two model boundaries and one tool boundary.

In the recorded demonstration, the user requests about $1,000 of Acme stock. The planning node generates a place_order call with symbol Acme and quantity 1000. The tool then sells those 1,000 units at $190 each. The trace preserves both the erroneous instruction and the tool’s execution of it; a final response alone would not expose that distinction.

The detailed node JSON adds model-version and sampling metadata alongside the inputs and outputs. Diagnosis proceeds backward from the consequential action: inspect the sell tool’s input, confirm that it received quantity 1000, and then inspect the preceding planner node. Its output already contains the wrong quantity. The tool did not invent the number; the planner converted the user’s monetary intent into the wrong tool argument.

8:058:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:05 · section reference included

Replay the bad decision against the repaired tool

Locating the bad tool call does not make future model outputs controllable. The repair moves enforcement to the tool boundary: a guardrail must reject an unacceptable order even when the planner requests it. The demonstration does not specify the stock guardrail’s exact rule, so the test establishes its response to this recorded failure rather than a general trading policy.

The same boundary that records a method can substitute its recorded output during a test. This lets you hold upstream decisions fixed while executing changed code:

  1. Load the trace of the failed run.
  2. Enable replay and choose which boundaries to stub.
  3. Stub the planning LLM so it emits the original incorrect order.
  4. Run the repaired place_order method against that order.
  5. Capture its result and assert that the order was blocked.

In the general regression strategy, unchanged nodes can all be stubbed. The essential choice is to keep the code under test live while preventing a fresh model decision from removing the failure condition.

A small Python harness expresses the assertion contract without depending on Chronicle’s entry-point syntax. Here, the tool adapter returns a dictionary with a status field, and the recorded planner arguments stay fixed:

python

from copy import deepcopy
from unittest.mock import Mock


def verify_recorded_order_is_blocked(repaired_tool):
    recorded_arguments = {
        "symbol": "Acme",
        "side": "sell",
        "quantity": 1000,
    }
    planner = Mock(return_value=deepcopy(recorded_arguments))

    order = planner("Sell about $1,000 of Acme stock")
    result = repaired_tool(**order)

    assert order == recorded_arguments
    assert result["status"] == "blocked"
    return result

The tool still receives the bad quantity. The test succeeds because the repaired boundary refuses it, not because the planner produced a better answer. A live boundary executes real code and its side effects, so a destructive tool needs gated execution or a sandbox; replay mode itself is not a safety barrier.

The displayed replay stubs the first LLM, preserving its recorded input and output, while leaving the tool and downstream LLM live. The tool returns a blocked result and the blocked-order assertion passes. This is selective replay, not the fully stubbed configuration with no model calls. It verifies the repaired tool against the original bad decision while still allowing later response generation to run.

Terminal displays cut-point test boundary results and four PASS checks: order blocked, no shares sold, agent1 stubbed, and place_order ran live.Open full source frame
Replay test blocks the order with the first agent stubbed and the tool and second agent live.
9:5610:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:56 · section reference included

Test enforcement and behavior separately

A recorded session can become a test case, but that does not make every aspect of agent quality a deterministic assertion. The two testing approaches answer complementary questions:

Test typeTargetMethod
Deterministic regressionGuardrails and tool logicRecorded inputs, live changed code, assertions
Behavioral evaluationTone and trajectory qualityAssess generated behavior, potentially with an LLM judge

For deterministic regression, recorded model outputs hold the surrounding context fixed. Stubbing every model boundary removes sampling from the test and avoids model calls and their associated inference charges; ordinary execution costs remain. Behavioral evaluation instead asks whether the agent’s generated response or chosen trajectory is appropriate. The speakers propose LLM-as-judge techniques for those more subjective assessments. A passing tool assertion protects a specific invariant, while behavioral evaluation examines the quality of the decisions and responses around it.

12:0812:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:08 · section reference included

Keep the execution envelope as a test case

Production debugging should not depend on bitwise identity from a hosted stack you do not control. Record the variables that explain the session: the LLM version, build ID, retrieved RAG chunks, and the inputs and outputs at the relevant boundaries. A prompt is only one ingredient in the resulting action. Capturing the full execution envelope preserves the context needed to understand why that action occurred.

Use that envelope to investigate the failure, run the repaired code against it, and retain it as a regression test. The speakers’ closing recommendation is to preserve generation-time variation rather than treat temperature zero as a reliability fix; their closing QR offers Chronicle code and related articles. For the stock agent, the durable improvement is that the recorded bad order remains available to test the tool that must stop it, even if the model never generates that exact mistake again.

Complete five-item takeaway slide: stop chasing bitwise determinism through the API; pin every variable against the session; capture the full envelope at the boundary; replay, fix, and keep the envelope as a test case; keep generation-time variation alive.Open full source frame
Five takeaways cover session variables, full-envelope capture, replay, retained tests, and generation-time variation.
13:1413:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:14 · section reference included

Resources

From the talk

  • Python tooling for recording method boundaries and testing changed code against recorded agent decisions, with examples and replay assertions.

Transcript download
Full transcript

Loading transcript…