AI Engineer World's Fair 2025
Stateful environments for vertical agents — Josh Purtell, Synth Labs
Loading slides…
Read the talk
Stateful environments for vertical agents
Separate an agent’s decisions from the application state it changes, so models can be replaced, work can be shared, and failed trajectories can be rolled back.
From a talk by Josh Purtell
Before you start: Familiarity with tool-calling agents and the distinction between application state and model context will help.
Where should an agent’s working state live?
When an agent works in finance, accounting or health, where should the application’s state and task logic live? Putting everything inside the agent makes those concerns difficult to separate as the system grows. Josh Purtell, introducing himself as Synth’s founder, calls the alternative a stateful environment: a workspace that captures state for the agent, with application logic organized separately from its decision-making.
This separation has a long history in reinforcement learning. The task specifies what the AI should accomplish without prescribing every action; the environment contains the task’s logic, while the learning algorithm decides how to act. RL-Glue was an early standard interface for that separation, followed in Purtell’s account by OpenAI Gym. These are precedents for the architecture, rather than a claim that RL-Glue originated all environment interfaces.
Software engineering brings that idea closer to a vertical application. SWE-bench provides an evaluation framework for resolving repository issues; SWE-agent develops the agent-computer interface through which an agent navigates, edits and executes code. The shared idea is a contained workspace with an interface suited to the AI using it. Stateful environments build on that lineage.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From answering a request to developing an artifact
A calculator call or a weather search needs little persistent workspace. Purtell contrasts those early language-model tools with richer API access, including interest in the Model Context Protocol. As tools become more capable, the agent can do more than return an answer: it can change something that subsequent actions depend on.
Purtell identifies Claude 3.5 Sonnet as a capability milestone for this shift. An agent can work on a product or artifact, inspect it, and improve it over successive steps. Claude Artifacts makes that persistent object visible in a web application. This is a progression in kinds of work, rather than release order: Sonnet and Artifacts arrived in June 2024, before MCP’s November announcement. The architectural pressure comes from maintaining a coherent object across a long sequence of actions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Compute outside the agent, expose what it needs
The environment computes results outside the agent implementation. The agent requests a change; the environment owns the underlying work. That work might call an API, manipulate an Excel document, or compute against an external source of truth whose results enter a system of record. The model’s decision and the application’s computation become separate responsibilities.
Excel makes the interface problem concrete. Giving an agent the entire application exposes far more surface area than a particular task may require. Instead, an environment can expose a representation that the agent can observe and manipulate usefully. Purtell compares this to showing an agent the relevant terminal view instead of the whole operating system: the interface should make the required work accessible without exposing everything underneath.
A network boundary extends this separation beyond code organization. The agent and its environment can run in different processes, which is useful for reinforcement-learning training and for multiple agents interacting with a workspace. The agent-facing representation, the computation behind it, and the process hosting that computation need not be the same thing.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recover state, then explore alternatives
Once working state has an explicit owner, reset and rollback become environment operations. A coding agent that derails can return to an earlier workspace instead of continuing from its accumulated mistakes. Recovery concerns the thing the agent changed, not just the messages describing its actions.
A small Python example makes that distinction concrete for the spreadsheet case. Here, the environment owns two cells and computes their total. It saves a checkpoint before an exploratory edit, then restores the cells after abandoning that edit.
python
from copy import deepcopy
class SheetEnvironment:
def __init__(self):
self.cells = {"A1": 100, "A2": 50}
def observe(self):
return {"cells": dict(self.cells),
"total": sum(self.cells.values())}
def set_cell(self, cell, value):
self.cells[cell] = value
return self.observe()
def snapshot(self):
return deepcopy(self.cells)
def restore(self, checkpoint):
self.cells = deepcopy(checkpoint)
return self.observe()
sheet = SheetEnvironment()
checkpoint = sheet.snapshot()
trial = sheet.set_cell("A2", 500)
restored = sheet.restore(checkpoint)
The trial state has A2 = 500; the restored state has A2 = 50, with A1 preserved. For a real environment, the checkpoint must cover the state needed for recovery. Restoring local cells does not automatically reverse a write already sent to an external system of record.
Recovery also enables search. Purtell points to Language Agent Tree Search, or LATS, and attributes its production implementation difficulty to missing environment abstractions. His description of getting it “for free” with a resettable environment is shorthand for removing a major obstacle: exploring an alternative requires returning to a comparable starting state. LATS still needs its search procedure, value estimation, reflection and environmental feedback; reset alone does not choose a good trajectory.
The Minecraft example shows how that capability is used:
- Start alternative trajectories from a shared point.
- Let the agent act along two branches.
- Compare the resulting progress.
- Select the better branch and continue from its state.
Purtell describes one branch as doing substantially better. The displayed terminal comparison shows Branch 1 and Branch 2, their action sequences and achievements, and a prompt asking which branch to continue. The useful result is the ability to retain a promising trajectory without remaining committed to the other one.
Purtell describes Minecraft tasks spanning hundreds or thousands of steps as a setting where rollback can help prevent derailment. That is his motivation for the example, rather than a measured trajectory length for the displayed run. The same need appears whenever an agent must sustain useful work over a long horizon: a bad stretch of actions should not force it to abandon everything that came before.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Finding environment implementations
Purtell closes by pointing builders to GitHub with the search phrase Synth AI Environments. He describes an open-source repository containing these abstractions and implementations across academic benchmarks—a place to examine how the environment boundary is expressed in working task interfaces.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The 2009 paper explains the separation of agent, environment and experiment programs, including communication across process boundaries.
The historical introduction of Gym's common toolkit and environments for reinforcement-learning experiments.
Research on interfaces that help language agents navigate repositories, edit files and execute tests.
The original benchmark for resolving real GitHub issues by modifying existing codebases.
Anthropic's June 2024 announcement introduces Artifacts as a workspace for iteratively developing generated content.
The original introduction to MCP's client/server approach for connecting AI applications with tools and data.
Combines Monte Carlo Tree Search with language-model reasoning, reflection and environmental feedback.
Further reading
Official experiments, prompts and setup instructions for programming, HotPotQA and WebShop.
Full transcript
Loading transcript…