# Stateful environments for vertical agents — Josh Purtell, Synth Labs

Josh Purtell

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/5rMc-moNVx0-stateful-environments-vertical-agents) · [Video](https://www.youtube.com/watch?v=5rMc-moNVx0)

## Stateful environments for vertical agents

Separate an agent’s decisions from the application state it changes, so models can be replaced, work can be shared, and failed trajectories can be rolled back.

Familiarity with tool-calling agents and the distinction between application state and model context will help.

### Where should an agent’s working state live?

[0:16](https://www.youtube.com/watch?v=5rMc-moNVx0&t=16s)

When an agent works in finance, accounting or health, where should the application’s state and task logic live? Putting everything inside the agent makes those concerns difficult to separate as the system grows. Josh Purtell, introducing himself as Synth’s founder, calls the alternative a **stateful environment**: a workspace that captures state for the agent, with application logic organized separately from its decision-making.

This separation has a long history in reinforcement learning. The task specifies what the AI should accomplish without prescribing every action; the environment contains the task’s logic, while the learning algorithm decides how to act. [RL-Glue](https://www.jmlr.org/papers/v10/tanner09a.html) was an early standard interface for that separation, followed in Purtell’s account by [OpenAI Gym](https://openai.com/index/openai-gym-beta/). These are precedents for the architecture, rather than a claim that RL-Glue originated all environment interfaces.

Software engineering brings that idea closer to a vertical application. [SWE-bench](https://arxiv.org/abs/2310.06770) provides an evaluation framework for resolving repository issues; [SWE-agent](https://arxiv.org/abs/2405.15793) develops the **agent-computer interface** through which an agent navigates, edits and executes code. The shared idea is a contained workspace with an interface suited to the AI using it. Stateful environments build on that lineage.

![Slide titled “What is an Environment?” lists RL-Glue, OpenAI Gym, and Agent-Computer Interface, with the speaker inset below.](https://ai.engineer/api/talk-essay-media/265c0079bd9a1b0402c918583e4a7bd25501746ff018da7476469b6717fc7d02)

Environment precedents: RL-Glue, OpenAI Gym, and Agent-Computer Interface.

### From answering a request to developing an artifact

[1:53](https://www.youtube.com/watch?v=5rMc-moNVx0&t=113s)

A calculator call or a weather search needs little persistent workspace. Purtell contrasts those early language-model tools with richer API access, including interest in the [Model Context Protocol](https://www.anthropic.com/news/model-context-protocol). As tools become more capable, the agent can do more than return an answer: it can change something that subsequent actions depend on.

Purtell identifies [**Claude 3.5 Sonnet**](https://www.anthropic.com/news/claude-3-5-sonnet) as a capability milestone for this shift. An agent can work on a product or artifact, inspect it, and improve it over successive steps. Claude Artifacts makes that persistent object visible in a web application. This is a progression in kinds of work, rather than release order: Sonnet and Artifacts arrived in June 2024, before MCP’s November announcement. The architectural pressure comes from maintaining a coherent object across a long sequence of actions.

### Compute outside the agent, expose what it needs

[3:03](https://www.youtube.com/watch?v=5rMc-moNVx0&t=183s)

**The environment computes results outside the agent implementation.** The agent requests a change; the environment owns the underlying work. That work might call an API, manipulate an Excel document, or compute against an external source of truth whose results enter a system of record. The model’s decision and the application’s computation become separate responsibilities.

Excel makes the interface problem concrete. Giving an agent the entire application exposes far more surface area than a particular task may require. Instead, an environment can expose a representation that the agent can observe and manipulate usefully. Purtell compares this to showing an agent the relevant terminal view instead of the whole operating system: the interface should make the required work accessible without exposing everything underneath.

A **network boundary** extends this separation beyond code organization. The agent and its environment can run in different processes, which is useful for reinforcement-learning training and for multiple agents interacting with a workspace. The agent-facing representation, the computation behind it, and the process hosting that computation need not be the same thing.

### Keep the application stable while agents change

[4:12](https://www.youtube.com/watch?v=5rMc-moNVx0&t=252s)

The first benefit is easier agent replacement. When a new model arrives, the prompts, reasoning strategy or agent implementation may need substantial revision. Keeping vertical application logic behind the environment interface lets that logic stay fixed during the change. If domain rules and agent behavior are interwoven, replacing the agent also becomes an application refactor.

![Slide titled “What Does This Get Us” lists Easy Agent Changes, Multi-agent, Reproducibility, and LATS!](https://ai.engineer/api/talk-essay-media/235bfbe3764441e95355d447f2f9289479a523428be3386573f25198ea1380b7)

Easy agent changes, multi-agent support, reproducibility, and LATS.

The same boundary supports multiple agents working on one persistent product over time. A separate environment process gives newly started agents or models a common place to direct their work. Purtell presents standard network boundaries as the foundation for reliable asynchronous production work. The boundary creates a place to manage coordination; the talk does not specify a conflict-resolution or concurrency protocol.

### Recover state, then explore alternatives

[4:59](https://www.youtube.com/watch?v=5rMc-moNVx0&t=299s)

Once working state has an explicit owner, **reset and rollback** become environment operations. A coding agent that derails can return to an earlier workspace instead of continuing from its accumulated mistakes. Recovery concerns the thing the agent changed, not just the messages describing its actions.

A small Python example makes that distinction concrete for the spreadsheet case. Here, the environment owns two cells and computes their total. It saves a checkpoint before an exploratory edit, then restores the cells after abandoning that edit.

```python
from copy import deepcopy

class SheetEnvironment:
    def __init__(self):
        self.cells = {"A1": 100, "A2": 50}

    def observe(self):
        return {"cells": dict(self.cells),
                "total": sum(self.cells.values())}

    def set_cell(self, cell, value):
        self.cells[cell] = value
        return self.observe()

    def snapshot(self):
        return deepcopy(self.cells)

    def restore(self, checkpoint):
        self.cells = deepcopy(checkpoint)
        return self.observe()

sheet = SheetEnvironment()
checkpoint = sheet.snapshot()
trial = sheet.set_cell("A2", 500)
restored = sheet.restore(checkpoint)
```

The trial state has `A2 = 500`; the restored state has `A2 = 50`, with `A1` preserved. For a real environment, the checkpoint must cover the state needed for recovery. Restoring local cells does not automatically reverse a write already sent to an external system of record.

Recovery also enables search. Purtell points to [Language Agent Tree Search](https://arxiv.org/abs/2310.04406), or **LATS**, and attributes its production implementation difficulty to missing environment abstractions. His description of getting it “for free” with a resettable environment is shorthand for removing a major obstacle: exploring an alternative requires returning to a comparable starting state. LATS still needs its search procedure, value estimation, reflection and environmental feedback; reset alone does not choose a good trajectory.

The Minecraft example shows how that capability is used:

1. Start alternative trajectories from a shared point.
2. Let the agent act along two branches.
3. Compare the resulting progress.
4. Select the better branch and continue from its state.

Purtell describes one branch as doing substantially better. The displayed terminal comparison shows Branch 1 and Branch 2, their action sequences and achievements, and a prompt asking which branch to continue. The useful result is the ability to retain a promising trajectory without remaining committed to the other one.

![Slide titled “Exploration for Long Horizons” shows terminal output for Branch 1 and Branch 2, including achievements, action sequences, and a branch-selection prompt.](https://ai.engineer/api/talk-essay-media/8e1142e306d9fe504ef919dc11a4163feb0bb09df5768a7d90733d58610a947d)

Two branch results and a prompt to choose which branch to continue.

Purtell describes Minecraft tasks spanning hundreds or thousands of steps as a setting where rollback can help prevent derailment. That is his motivation for the example, rather than a measured trajectory length for the displayed run. The same need appears whenever an agent must sustain useful work over a long horizon: a bad stretch of actions should not force it to abandon everything that came before.

### Finding environment implementations

[6:23](https://www.youtube.com/watch?v=5rMc-moNVx0&t=383s)

Purtell closes by pointing builders to GitHub with the search phrase `Synth AI Environments`. He describes an open-source repository containing these abstractions and implementations across academic benchmarks—a place to examine how the environment boundary is expressed in working task interfaces.

## Resources

- [RL-Glue](https://jmlr.csail.mit.edu/papers/volume10/tanner09a/tanner09a.pdf) — The 2009 paper explains the separation of agent, environment and experiment programs, including communication across process boundaries.
- [OpenAI Gym's original announcement](https://openai.com/index/openai-gym-beta/) — The historical introduction of Gym's common toolkit and environments for reinforcement-learning experiments.
- [SWE-agent and agent-computer interfaces](https://arxiv.org/abs/2405.15793) — Research on interfaces that help language agents navigate repositories, edit files and execute tests.
- [SWE-bench](https://arxiv.org/abs/2310.06770) — The original benchmark for resolving real GitHub issues by modifying existing codebases.
- [Claude 3.5 Sonnet and Artifacts](https://www.anthropic.com/news/claude-3-5-sonnet) — Anthropic's June 2024 announcement introduces Artifacts as a workspace for iteratively developing generated content.
- [Model Context Protocol announcement](https://www.anthropic.com/news/model-context-protocol) — The original introduction to MCP's client/server approach for connecting AI applications with tools and data.
- [Language Agent Tree Search](https://arxiv.org/abs/2310.04406) — Combines Monte Carlo Tree Search with language-model reasoning, reflection and environmental feedback.
- [Language Agent Tree Search code](https://github.com/lapisrocks/LanguageAgentTreeSearch) — Official experiments, prompts and setup instructions for programming, HotPotQA and WebShop.

## Slides

### Stateful Environments

Josh Purtell introduces stateful environments as a way to capture state for agents, drawing on patterns for vertical applications such as finance, accounting, and health.

[0:29](https://www.youtube.com/watch?v=5rMc-moNVx0&t=29s) [Synth AI title slide reading “Stateful Environments” above a circuit-like illustration.](https://assets.ai.engineer/sites/aiecode2025/sha256/3e017f3aee72d340df7e52e86bf998e92885b730e3737e0bc19ad21575a347e7.webp)

Stateful Environments

Synth AI
Date

Interface text:
AIE
AI Engineer World's Fair
Microsoft
smol ai

### What is an Environment?

Purtell traces environments from RL-Glue and OpenAI Gym to the agent-computer interface: ways to separate task logic from the AI algorithm and give agents a stateful workspace.

[0:50](https://www.youtube.com/watch?v=5rMc-moNVx0&t=50s) [Slide listing RL-Glue, OpenAI Gym, and Agent-Computer Interface.](https://assets.ai.engineer/sites/aiecode2025/sha256/49c5c89b7dc35312a62dda22319e1eb7269d641cfa6ebdcd573ca0b677fe7c3a.webp)

What is an Environment?
- RL-Glue
- OpenAI Gym
- Agent-Computer Interface

Interface text:
AIE
AI Engineer World’s Fair
aws

[1:08](https://www.youtube.com/watch?v=5rMc-moNVx0&t=68s) [Slide listing RL-Glue, OpenAI Gym, and the agent-computer interface.](https://assets.ai.engineer/sites/aiecode2025/sha256/f9ef5c2c6f0af4cfd864a9ca79498bcbe0f8dd76edd35eb3ad684a0e6068171f.webp)

What is an Environment?
- RL-Glue
- OpenAI Gym
- Agent-Computer Interface

Interface text:
AIE
AI Engineer World’s Fair
aws

### What’s the story so far?

Purtell traces tool use from simple calculations to APIs and Claude Artifacts, where agents can iterate on an artifact over a longer horizon. This motivates stateful abstractions.

[1:51](https://www.youtube.com/watch?v=5rMc-moNVx0&t=111s) [Slide listing in-memory grounding, API-based tool use, Claude Artifacts, and a question mark.](https://assets.ai.engineer/sites/aiecode2025/sha256/06bc6d7f6e76a3ba192e233ab83d087a535a907221bdc6d69d39fe7ad086a92e.webp)

What’s the story so far?
- In-Memory grounding
- API-based tool-use
- Claude Artifacts
- ???

Other visible text:
AIE
AI Engineer
World’s Fair
aws

### What is a Stateful Environment

A stateful environment computes results outside the agent and exposes a simplified view the agent can observe and manipulate. Network boundaries can let the agent and environment run in separate processes.

[3:01](https://www.youtube.com/watch?v=5rMc-moNVx0&t=181s) [Slide describing a stateful environment as an engine that computes results, with a simplified representation and often network boundaries.](https://assets.ai.engineer/sites/aiecode2025/sha256/f08e61e5bdc296a14ec680a039b2e77a2138c4d61791d5b6f4c4903ebbf87787.webp)

What is a Stateful Environment
- Engine that computes results
- Simplified representation
- (Often) network boundaries

Interface text:
AIE
a​​ws
AI Engineer
World’s Fair

[3:07](https://www.youtube.com/watch?v=5rMc-moNVx0&t=187s) [Slide describing a stateful environment as an engine that computes results, with a simplified representation and often network boundaries.](https://assets.ai.engineer/sites/aiecode2025/sha256/d7fa9263b502c6c8151477c23e07cdddb31445d8aaa8301fc2bd308ae8a03a4a.webp)

What is a Stateful Environment
- Engine that computes results
- Simplified representation
- (Often) network boundaries

Interface text:
AIE
Microsoft
smol ai
AI Engineer
World’s Fair

### What Does This Get Us

Keeping app logic separate makes agents easier to change and lets multiple agents share an environment. Resetting the environment enables rollbacks and supports Language Agent Tree Search.

[4:12](https://www.youtube.com/watch?v=5rMc-moNVx0&t=252s) [Slide lists easy agent changes, multi-agent use, reproducibility, and LATS.](https://assets.ai.engineer/sites/aiecode2025/sha256/d0e91d475142aa4b7cfa9ed42dd4ab229c11e9e76b045dcdfde8b8cc9bc3aa2d.webp)

What Does This Get Us
- Easy Agent Changes
- Multi-agent
- Reproducibility
- LATS!

AIE
Microsoft
smol ai
AI Engineer
World’s Fair

### Exploration for Long Horizons

A tree-search screenshot compares two Minecraft branches: the first has three final achievements and the second has two. A prompt asks which branch to continue.

[5:25](https://www.youtube.com/watch?v=5rMc-moNVx0&t=325s) [Slide shows terminal results for two branches, including achievements and a prompt to choose which branch to continue.](https://assets.ai.engineer/sites/aiecode2025/sha256/e4b9ed754bf33680ec91c6fcaacca691f94862be23518503a957c704642e2069.webp)

Exploration for Long Horizons

Step 17: [+ACHIEVEMENT: ['collect_wood']]

📊 BRANCH RESULTS:

Branch 1:
  Final achievements: 3
  Achievement score: 0.133
  New achievements: ['collect_drink', 'collect_wood']
  Actions taken: right -> do -> right -> do -> right...

Branch 2:
  Final achievements: 2
  Achievement score: 0.133
  New achievements: ['collect_wood']
  Actions taken: right -> do -> right -> do -> right...

Choose branch to continue (1-2):

AIE
Microsoft
smol ai
AI Engineer World's Fair

### Stateful Environments

Purtell closes by pointing to Synth AI Environments, an open-source repository with implementations across academic benchmarks.

[6:37](https://www.youtube.com/watch?v=5rMc-moNVx0&t=397s) [Brown slide titled “Stateful Environments” with Synth AI branding and a gold branching circuit graphic.](https://assets.ai.engineer/sites/aiecode2025/sha256/70a4987b3f5f99694f809792eb12d500f16f6e83a37f0e37069d34b0883592ee.webp)

Stateful Environments

Synth AI
Date

Interface text:
AIE
aws
AI Engineer
World’s Fair

## Full transcript

[0:00](https://www.youtube.com/watch?v=5rMc-moNVx0&t=0s) [upbeat music] All right.

[0:16](https://www.youtube.com/watch?v=5rMc-moNVx0&t=16s) Hi, I'm Josh, founder of Synth. I help people make their agents a lot better. And over the last few months, uh, I found s-some patterns around structuring people's agent code that I think they've found very helpful and I've found very helpful for, you know, thinking about how to build effective agents, especially for vertical applications like finance,

[0:41](https://www.youtube.com/watch?v=5rMc-moNVx0&t=41s) accounting, health, um, and so on and so forth. So I like to call these stateful environments because they're environments that capture state for the agent.

[0:51](https://www.youtube.com/watch?v=5rMc-moNVx0&t=51s) So, um, let's define terms. What is an environment? Uh, it, it feels like a loose term, but it actually has quite a long history. People working on reinforcement learning tasks, which are really just tasks where you're trying to get the AI to do something, um, without stipulating how to do it, have been using environments to kinda containerize

[1:11](https://www.youtube.com/watch?v=5rMc-moNVx0&t=71s) the logic behind the task away from their AI algorithm for quite a while. So the first implementation was RL Glue, then OpenAI, back when OpenAI was an RL company and, and not really a language model company, came out with the OpenAI Gym.

[1:26](https://www.youtube.com/watch?v=5rMc-moNVx0&t=86s) And then most recently, probably the first kind of vertical-ish application in academic papers, uh, SWE-bench and SWE-agent, um, kinda coined the term of agent-computer interface. So people have been thinking about containerizing a kind of stateful, uh, workspace for AIs to have for, for quite a while.

[1:47](https://www.youtube.com/watch?v=5rMc-moNVx0&t=107s) This is not reinventing the wheel, we're just building on top of what o-people have already thought about.

[1:53](https://www.youtube.com/watch?v=5rMc-moNVx0&t=113s) Okay, um, so why are we adding on this abstraction of statefulness now? Well, two years ago, um, people mostly gave their LMs tools to calculate simple sums or, uh, you know, maybe search the internet for the weather.

[2:12](https://www.youtube.com/watch?v=5rMc-moNVx0&t=132s) You really didn't need to have a lot of clean, heavy-duty abstractions, um, for, for pretty simple logic like that. As models got better and people wanted them to use more effective tools, they moved to API-based tool use and, you know, maybe you see that with some people getting excited about MCP.

[2:30](https://www.youtube.com/watch?v=5rMc-moNVx0&t=150s) Um, and it wasn't really until models got a lot better with Sonnet 3.5 that people started, uh, kind of thinking about a wor- a, a product or an artifact that the AI works on and iterates on and improves step over step over a long horizon.

[2:47](https://www.youtube.com/watch?v=5rMc-moNVx0&t=167s) Um, and I think when Claude Artifacts came out is probably when a lot of people started thinking about having some abstractions to help agents like Claude work on product, um, and artifacts like Claude Artifacts in the web app.

[3:03](https://www.youtube.com/watch?v=5rMc-moNVx0&t=183s) So, um, that's kind of the impetus, the why now. Uh, so what are we contributing? Well, a stateful environment is an engine that computes results external to the agent implementation.

[3:16](https://www.youtube.com/watch?v=5rMc-moNVx0&t=196s) Um, the agent manipulates the environment somehow, but there's a lot of logic underneath that might involve, uh, accessing an API, working on, um, a-an Excel document or, or some kind of external, um, uh, you know, source of truth that gets computed on and goes into a, a system of record.

[3:36](https://www.youtube.com/watch?v=5rMc-moNVx0&t=216s) It can be a lot for an agent to interact with, uh, Excel though, like the entire application. So a stateful environment exposes a kind of representation or a version of that environment that the agent can, can make sense of, can observe, and, and manipulate, uh, usefully.

[3:52](https://www.youtube.com/watch?v=5rMc-moNVx0&t=232s) So you don't have to show the agent the whole, um, OS. You, you kind of just show it what it needs to see in the terminal. And then often, and this is important for people doing RL training, but it can also be really handy in multi-agent settings, network boundaries, um, so that your agent doesn't have to run

[4:07](https://www.youtube.com/watch?v=5rMc-moNVx0&t=247s) in the same process as whatever your stateful environment is.

[4:12](https://www.youtube.com/watch?v=5rMc-moNVx0&t=252s) Okay, um, so what does this get us? I help people improve their agents. Um, if you containerize the logic of your vertical app into code that never changes, it's a lot easier to completely revamp your agent when the new model comes out.

[4:27](https://www.youtube.com/watch?v=5rMc-moNVx0&t=267s) It's a lot harder to do that when all the logic is kind of just clumped together. Um, what else does it give you? Well, if you have a separate process de-determining your environment, um, that has standard network boundaries, you can easily have multi-agent and spin up new models, uh, or spin up new agents to work on this

[4:46](https://www.youtube.com/watch?v=5rMc-moNVx0&t=286s) single product together, um, across time and, and there's really no problems. People have thought about how to do asynchronous work, and the an- the answer to that question of how to do asynchronous work in a reliable way in production is network boundaries.

[4:59](https://www.youtube.com/watch?v=5rMc-moNVx0&t=299s) Um, and then I think the most exciting thing is once you have, um, this, this boundary, you can start doing things like, uh, resetting the state of the thing that your agent is working on.

[5:09](https://www.youtube.com/watch?v=5rMc-moNVx0&t=309s) You could do rollbacks. I think a lot of people working with agents in a co- in a code setting know how valuable it is to just be able to roll back the agent after it's kind of gotten derailed.

[5:19](https://www.youtube.com/watch?v=5rMc-moNVx0&t=319s) And if you have stateful environments, that's really easy to implement. And so in particular, um, the-- a few years ago, there was a paper called Language Agent Tree Search, um, that was, you know, really impressive and it got really good results, but it was almost impossible to implement in production because just nobody had really good, um, abstractions

[5:38](https://www.youtube.com/watch?v=5rMc-moNVx0&t=338s) for it. And, and techniques like this are really useful in a long-horizon setting, like a lot of builders care about today. Um, and if you have a resettable environment, you sort of get language agent tree search for free.

[5:50](https://www.youtube.com/watch?v=5rMc-moNVx0&t=350s) And so here's kind of a screenshot of, uh, a step in the tree search. The agent branched out in two directions while playing Minecraft. Um, one of those branches did a lot better, and then it's really easy to kind of just converge, pick the best branch, and go from there.

[6:06](https://www.youtube.com/watch?v=5rMc-moNVx0&t=366s) Um, and in a game like Minecraft where you have hundreds or thousands of steps, avoiding derailing and resetting like that can be really handy. Um, but maybe not just in Minecraft, also in, in kind of a lot of other settings where people are having their agents do a lot of work over long horizons.

[6:23](https://www.youtube.com/watch?v=5rMc-moNVx0&t=383s) Um, so if you'd like to see some implementations of stateful environments, you can go to our GitHub. There's an open source repository that captures a lot of these abstractions, and there's implementations across a lot of academic benchmarks.

[6:37](https://www.youtube.com/watch?v=5rMc-moNVx0&t=397s) Um, how do you find that? Look for Synth AI Environments. Um, and, and that's the talk. [upbeat music]
