AI Engineer World's Fair 2026

Your Agent Didn’t Fail. Your Harness Did.

Loading slides…

Read the talk

Your Agent Didn’t Fail. Your Harness Did.

A reply can reach the user while disappearing from durable history. Reliable agents need explicit state ownership, ordered mutations, bounded execution, scoped authority, and evidence of outcomes.

From a talk by Vinoth Govindarajan

Before you start: Familiarity with tool-calling agents, persisted conversation history, and basic concurrency will help you follow the failure mechanisms.

The refund was remembered—until the next turn

A user asks an assistant to remember a customer refund. The assistant confirms that it has recorded the fact for the next turn. The interface looks normal: the reply arrives, nothing crashes, and no error appears. But when the next turn begins, the system cannot reconstruct the refund. The user experienced success; the durable history has a hole.

Opening incident slide with a support chat beside a backend memory store marked “NOT PERSISTED” and “MISSING.”Open full source frame
The assistant confirms it remembered the refund, but the backend shows no persisted record.

A crash at least exposes a stopping boundary. An operator can often identify the last known good point and restart from there. Silent success offers no such signal. Delivery can succeed while persistence fails, leaving both the user and the operator without an obvious reason to investigate. The next answer may still sound confident because a model can reason coherently over incomplete history. Agent reliability therefore cannot stop at the quality of the model’s answer.

0:320:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:32 · section reference included

Model proposes, harness commits, receipt proves

Vinoth Govindarajan introduces this problem from his work on core data and AI infrastructure at OpenAI, following distributed-systems work at Apple and Uber. He also writes The Agent Stack. His examples use OpenClaw because its public issues, code, and documentation make the surrounding infrastructure visible; he is not presenting as an OpenClaw maintainer or pitching the product.

The model proposes, the harness commits, and the receipt proves it. A model may suggest a message, tool invocation, edit, or command. The harness—the runtime and control machinery around that model—owns the actual state transition, authority check, and ordered commit. The receipt is the evidence that survives after the turn ends.

Three stages connected by arrows: Model—Proposes, Harness—Commits, and Receipt—Proves.Open full source frame
The production contract: Model proposes, Harness commits, Receipt proves.

That contract has three obligations: own the state, order the mutation, and prove the action. Each fact needs a named owner and a replay path. Shared mutable state needs an ordered commit path. A transcript records what the agent said; a receipt records what the system allowed, attempted, executed, and confirmed at the user-visible boundary. Those records answer different questions.

The distinction resembles the difference between a car’s engine and its control systems. Horsepower matters, but so do steering, brakes, road rules, the dashboard, and the black box. The model supplies capability. The harness supplies control over where that capability goes and evidence of what happened along the way.

2:012:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:01 · section reference included

Every turn starts with an assembled working set

Personal agents and coding agents share a useful architectural pattern. Events arrive through chat, webhooks, timers, heartbeats, or other external systems. A control plane maps each event to a session key, which determines its state boundary. A session lane provides one active writer for that mutable state. The runtime invokes models and tools, approvals and policies constrain tool actions, and the audit trail supplies the run receipt. The path is event → session key → throttled execution → tools → audit.

The model does not remember that session in the human sense. For each turn, the harness rebuilds a working set from the transcript, session state, memory, policy, and tool definitions. The model sees only what this assembly process supplies. A stale memory snapshot or missing transcript entry can change the answer without making its prose any less fluent. Coherence does not prove that the working set was complete.

Timeouts, retries, idempotency, logs, ordering, and state ownership are familiar distributed-systems concerns. Agents place those concerns around a probabilistic planner that can change its plan, reconstructs context on every turn, accepts more kinds of events, and acts through more external surfaces. The failure mechanisms are familiar; the expanding combinations make them easier to trigger and harder to explain.

4:114:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:11 · section reference included

Delivered is not remembered

The first failure shape returns to the opening incident. In the Telegram state-hole case Govindarajan describes, a routed reply could reach the user without the turn being written to the active context or transcript. The send succeeded and the logs looked healthy, but the next turn had no durable record of the exchange. A successful send proved transport, not future context. The missing boundary was state ownership.

An owner here is a system of record, not a person. Its persisted state determines what the system can later reconstruct.

FactAuthoritative owner
Calendar eventCalendar system
Support statusTicketing system
Code changeWorkspace or repository
Conversation turnSession transcript
User preferenceMemory store

Storage identifies where the bytes live. Ownership identifies which persisted record establishes the fact.

For a reply to become reliable memory, the system must persist the turn, identify its owner, and make replay possible. Ask this for every fact the agent might use later: which system owns it, and how does a subsequent turn recover it? If no owner can replay the fact, the system has not reliably remembered it, regardless of what the assistant told the user.

6:186:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:18 · section reference included

Two correct writes can erase each other

Once state has an owner, the next question is who can change it and in what order. In the overlapping-writer incident, two callers load the same old state and modify different records. Each operation is locally correct. But each saves a whole snapshot, so the second save silently erases the first caller’s change. A dismissed commitment can return, or a follow-up can arrive twice. The failure is in the load–modify–save boundary, not in either writer’s intended change.

A small TypeScript example makes the lost update concrete. Here, one caller dismisses a refund follow-up while another completes an invoice follow-up. Both start from the same snapshot:

typescript

type Status = "pending" | "dismissed" | "completed";
type Commitments = Record<string, Status>;

let stored: Commitments = {
  refundFollowUp: "pending",
  invoiceFollowUp: "pending",
};

const callerA = structuredClone(stored);
const callerB = structuredClone(stored);

callerA.refundFollowUp = "dismissed";
callerB.invoiceFollowUp = "completed";

stored = structuredClone(callerA);
stored = structuredClone(callerB);

// The second snapshot restores the old refund status.
console.log(stored);
// { refundFollowUp: "pending", invoiceFollowUp: "completed" }

The records are separate, but the saved object is shared. Ordering only the final assignments would not repair the stale snapshot: the mutation path must prevent a save from overwriting changes made since its read.

One mutable state boundary needs one ordered commit path. This does not require serializing the whole agent system. Subagents can fan out, reads and independent retrieval can run in parallel, and many sessions can execute at once. A queue, mutex, transaction, or lock can protect the relevant mutation boundary; its scope must match the state being shared. Keep unrelated work concurrent and serialize the work necessary to preserve that state.

Users experience these ordering defects as behavior. A lost correction feels forgetful. A stuck session lane feels dead. Completion announced before delivery feels confused. Ordering is therefore a product property as well as a storage concern: it determines whether the agent behaves consistently with what the user just did.

8:118:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:11 · section reference included

Silence needs an ending

A session can contain a tool call without a matching tool result. The process may have died, the connection may have dropped, or a timeout may have occurred before the result was recorded. In the dangling-call failure described here, the run continues waiting for an event that cannot arrive. New messages queue behind it, and the user sees an agent that has stopped responding. The original cause matters for debugging; the missing lifecycle boundary explains why the failure persists.

Bound the wait at every level:

  • Runs: deadlines limit duration, cancellation provides an exit, and watchdogs make stuck work visible.
  • Tools: timeouts and explicit error results prevent missing responses from becoming indefinite waits.
  • Channels: recovery commands need a path that does not queue behind the blocked work they are supposed to repair.

Every external boundary needs a terminal outcome: success, failure, timeout, cancellation, or exhaustion of the allowed attempts. Record that outcome in the receipt so the next step does not have to infer whether the work is still alive.

“Bound every wait” slide with cards listing run deadlines, cleanup and terminal state; tool timeouts, error handling and max attempts; and channel heartbeat, retry and idempotency.Open full source frame
Bound every wait across runs, tools, and channels.
9:5510:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:55 · section reference included

A button click is not lasting authority

Once a conversation can cause actions, the harness must distinguish the ability to request an action from permission to execute it. In the approval-drift incident, an expired approval callback was treated as retryable. The stale callback remained durable, survived restarts, and blocked later channel work. A button click existed in the past, but valid authority no longer existed. The failure was a persistent retry loop, not evidence that an unauthorized action executed.

Approval is scoped execution state. It must remain attached to the particular action it authorized, and expiration must terminate that state rather than create another retry. A useful approval record identifies the approver, session, run, tool, exact arguments, lifetime, outcome, and associated receipt. If those bindings disappear during a retry, replay, or channel callback, the harness can no longer establish that the action being executed is the action that was approved.

Least privilege narrows the available tool surface. Scoped credentials determine which identity an action uses. Approval governs whether execution may proceed, and audit preserves evidence afterward. The model can reason about these boundaries and request an action, but the system must enforce the decision.

11:1911:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:19 · section reference included

Tool success is not proof of delivery

The final incident reverses the opening failure. There, the user saw a reply that durable state did not retain. Here, an internal component reports success but the user sees nothing. In the web-chat or TUI case described in the talk, the message tool reported success even though its message did not render; ordinary assistant replies still appeared. The internal path had accepted the request, but acceptance did not establish the user-visible result. The agent could later insist it had already sent something that the user had never seen.

Proof is a chain. The model proposes an action, policy allows or denies it, execution attempts it, and the user-visible edge confirms—or fails to confirm—the outcome. The receipt preserves those distinctions.

RecordWhat it establishes
TranscriptWhat the agent said
Tool resultWhat one component reported
ReceiptEvidence across the relevant boundaries

A receipt should preserve the absence of confirmation as well as confirmation itself. Otherwise an internal success report can turn into an unsupported claim about the external world.

13:0013:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:00 · section reference included

Trace one real production path

The five failure shapes—state holes, overlapping writers, dangling tool calls, approval drift, and missing edge proof—can all be examined through the same questions: what did the user see, which boundary broke, and what would a receipt have exposed? Start with one agent system and one real production path. Ask what woke it up, what state it inherited, which authority it used, what executed, and what evidence survived. These questions turn a fluent conversation into an inspectable causal chain.

Five cards ask “What woke it up?”, “Which state?”, “Which authority?”, “What executed?”, and “What survived?”Open full source frame
Five questions for every run: trigger, state, authority, execution, and surviving evidence.

Walk that path in order:

  1. Identify the trigger. Was it a user message, webhook, timer, tool result, subagent, or replay? Record both the trigger type and its identity. Without identity, deduplication, ordering, and authorization become difficult to reconstruct.
  2. Capture the inherited state. Identify the transcript, session state, memory snapshot, policy version, and tool surface supplied to the run. This is the working set the model actually reasoned over.
  3. Bind the authority. Record the actor, session, run, tool, arguments, scope, and lifetime. Permission should attach to one pending action, not float across the conversation.
  4. Record execution. Preserve the actual tool or API call, its arguments, attempt number, idempotency key, and external result. This is evidence about side effects, not a summary of intentions.
  5. Verify the outcome. Check the boundary the user cares about: the updated ticket, rendered message, changed file, or existing calendar event. The receipt should reach that boundary.
14:0814:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:08 · section reference included

Reliability has to survive the turn

Apply the audit to the refund example. A user message woke the agent. A channel send executed. Delivery evidence survived, but the durable conversation turn did not. That locates the failure at the state boundary: the system could prove that it sent a reply, but could not reconstruct the exchange for the next turn. A better prompt or a more capable model would not repair the missing persistence.

Owning state, ordering mutations, and proving actions are obligations across turns. Better models can improve the reasoning inside a turn; ownership, ordering, lifecycle, authority, and evidence determine whether the surrounding system remains reliable afterward. Once text can become action, the production question includes whether the system can bound the work, constrain its authority, and preserve evidence of its effects. An answering loop is only part of that system.

Govindarajan closes by recommending the OpenAI Agents SDK as a starting point for building agents and The Agent Stack for further reading on production systems. The SDK recommendation is a starting point, not a guarantee that every application automatically satisfies the ownership, ordering, authority, and delivery contract described here. Those boundaries still need to be established for the system being built.

16:1816:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:18 · section reference included

Resources

From the talk

  • Vinoth Govindarajan's publication, recommended in the talk for further reading on production agent systems.

Updates since the talk

  • Current Python SDK guidance for storing conversation history, resuming interrupted runs and choosing a persistence backend, including compaction concurrency limitations.

Transcript download
Full transcript

Loading transcript…