AI Engineer World's Fair 2025

Stateful environments for vertical agents — Josh Purtell, Synth Labs

Loading slides…

Read the talk

Stateful environments for vertical agents

Separate an agent’s decisions from the application state it changes, so models can be replaced, work can be shared, and failed trajectories can be rolled back.

From a talk by Josh Purtell

Before you start: Familiarity with tool-calling agents and the distinction between application state and model context will help.

Where should an agent’s working state live?

When an agent works in finance, accounting or health, where should the application’s state and task logic live? Putting everything inside the agent makes those concerns difficult to separate as the system grows. Josh Purtell, introducing himself as Synth’s founder, calls the alternative a stateful environment: a workspace that captures state for the agent, with application logic organized separately from its decision-making.

This separation has a long history in reinforcement learning. The task specifies what the AI should accomplish without prescribing every action; the environment contains the task’s logic, while the learning algorithm decides how to act. RL-Glue was an early standard interface for that separation, followed in Purtell’s account by OpenAI Gym. These are precedents for the architecture, rather than a claim that RL-Glue originated all environment interfaces.

Software engineering brings that idea closer to a vertical application. SWE-bench provides an evaluation framework for resolving repository issues; SWE-agent develops the agent-computer interface through which an agent navigates, edits and executes code. The shared idea is a contained workspace with an interface suited to the AI using it. Stateful environments build on that lineage.

Slide titled “What is an Environment?” lists RL-Glue, OpenAI Gym, and Agent-Computer Interface, with the speaker inset below.Open full source frame
Environment precedents: RL-Glue, OpenAI Gym, and Agent-Computer Interface.
0:160:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:16 · section reference included

From answering a request to developing an artifact

A calculator call or a weather search needs little persistent workspace. Purtell contrasts those early language-model tools with richer API access, including interest in the Model Context Protocol. As tools become more capable, the agent can do more than return an answer: it can change something that subsequent actions depend on.

Purtell identifies Claude 3.5 Sonnet as a capability milestone for this shift. An agent can work on a product or artifact, inspect it, and improve it over successive steps. Claude Artifacts makes that persistent object visible in a web application. This is a progression in kinds of work, rather than release order: Sonnet and Artifacts arrived in June 2024, before MCP’s November announcement. The architectural pressure comes from maintaining a coherent object across a long sequence of actions.

1:532:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:53 · section reference included

Compute outside the agent, expose what it needs

The environment computes results outside the agent implementation. The agent requests a change; the environment owns the underlying work. That work might call an API, manipulate an Excel document, or compute against an external source of truth whose results enter a system of record. The model’s decision and the application’s computation become separate responsibilities.

Excel makes the interface problem concrete. Giving an agent the entire application exposes far more surface area than a particular task may require. Instead, an environment can expose a representation that the agent can observe and manipulate usefully. Purtell compares this to showing an agent the relevant terminal view instead of the whole operating system: the interface should make the required work accessible without exposing everything underneath.

A network boundary extends this separation beyond code organization. The agent and its environment can run in different processes, which is useful for reinforcement-learning training and for multiple agents interacting with a workspace. The agent-facing representation, the computation behind it, and the process hosting that computation need not be the same thing.

3:033:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:03 · section reference included

Keep the application stable while agents change

The first benefit is easier agent replacement. When a new model arrives, the prompts, reasoning strategy or agent implementation may need substantial revision. Keeping vertical application logic behind the environment interface lets that logic stay fixed during the change. If domain rules and agent behavior are interwoven, replacing the agent also becomes an application refactor.

Slide titled “What Does This Get Us” lists Easy Agent Changes, Multi-agent, Reproducibility, and LATS!Open full source frame
Easy agent changes, multi-agent support, reproducibility, and LATS.

The same boundary supports multiple agents working on one persistent product over time. A separate environment process gives newly started agents or models a common place to direct their work. Purtell presents standard network boundaries as the foundation for reliable asynchronous production work. The boundary creates a place to manage coordination; the talk does not specify a conflict-resolution or concurrency protocol.

4:124:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:12 · section reference included

Finding environment implementations

Purtell closes by pointing builders to GitHub with the search phrase Synth AI Environments. He describes an open-source repository containing these abstractions and implementations across academic benchmarks—a place to examine how the environment boundary is expressed in working task interfaces.

6:236:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:23 · section reference included

Resources

From the talk

  • The 2009 paper explains the separation of agent, environment and experiment programs, including communication across process boundaries.

  • The historical introduction of Gym's common toolkit and environments for reinforcement-learning experiments.

  • Research on interfaces that help language agents navigate repositories, edit files and execute tests.

  • SWE-benchPaper1:26

    The original benchmark for resolving real GitHub issues by modifying existing codebases.

  • Anthropic's June 2024 announcement introduces Artifacts as a workspace for iteratively developing generated content.

  • The original introduction to MCP's client/server approach for connecting AI applications with tools and data.

  • Combines Monte Carlo Tree Search with language-model reasoning, reflection and environmental feedback.

Transcript download
Full transcript

Loading transcript…