AI Engineer World's Fair 2026

I Run a Fleet of AI Agents Across Three Machines. Here's What Broke.

Loading slides…

Read the talk

I Run a Fleet of AI Agents Across Three Machines. Here's What Broke.

Kyle Jaejun Lee’s agent fleet turns attention, state, approvals and machine placement into separate engineering problems—and exposes what remains unsolved between them.

From a talk by Kyle Jaejun Lee

Before you start: Familiarity with terminal sessions, Git and AI coding agents will help; no Kubernetes experience is required.

The first bottleneck was attention

A MacBook sleeps. Long-running coding jobs need somewhere that does not. Kyle Jaejun Lee’s everyday fleet spans that laptop and two headless, always-on Linux boxes, connected through one control plane. The MacBook handles heavy coding and personal projects; Linux A takes long-running coding tasks, while Linux B runs short-lived personal projects. He opens with the fleet running live: daily infrastructure across two operating systems.

Before distributing the work, though, Lee ran into a limit on one machine: his own attention. A few tmux panes became four, five, then six simultaneous agent contexts. Every additional agent gave him three jobs: scheduler, deciding who should do what; memory, remembering what each agent was doing; and reviewer, checking the results. Six active contexts were more than he could keep in his head.

The overload diagram shows a box labeled me containing scheduler, memory and reviewer, pointing to six context boxes.Open full source frame
One human fills three roles across multiple live contexts.
0:000:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Separate context, then persist it

How can a handful of executives oversee thousands of people? They do not retain everyone’s working context. Lee applied that separation to his agents, building a hierarchy with four actual entity types: CEO, VP, manager and worker. Each entity is an agent with its own scoped context and approval boundary. Context flows downward in the slices needed for each task; results flow upward, and Lee reviews what reaches the top.

Lee describes his supervisory context load falling from six contexts to one. That is his account of what he has to hold in mind, rather than a measure of fleet throughput. The hierarchy reduces the context reaching the human, but each agent still has a finite context window. The next question is where its state should live when that window fills.

His answer is a workspace on disk for every entity. The layout separates common obligations, machine-specific information and the work an individual agent needs to continue:

Location or contentPurpose
sharedContext every entity must honor
machinesMachine-bound state
Entity workspaceThe entity’s mission and current status
Handoff folderWork product passed to the next agent or session

The durable state lives in files. A model’s current conversation is no longer the only place that knows what the agent is doing.

That changes how Lee handles a full context window. He finds compaction slow and dislikes leaving retention decisions to a summary, so inside Claude he clears the context completely. The fresh context then reads the handoff and history files the agent wrote for itself and resumes from those records. Current Claude Code session guidance does allow focus instructions to steer compaction, although the summary remains lossy; that does not establish what Lee’s installed version supported. His preference is to make the continuity material explicit before resetting.

A context wipe—or even a machine crash—can therefore leave the written work available for recovery. This preserves recorded state and work product; it does not keep a running process alive through a crash. With continuity moved onto disk, the next problem is whether agents remain aligned with the intended plan.

1:281:36
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:28 · section reference included

Make approval a blocking operation

Plans drift as they move through the hierarchy. Initially, Lee detected that drift by entering individual panes and inspecting the work. He replaced those visits with a review gateway: any layer that wants to act submits its plan and blocks. The plan is pending until he approves it; approval fires a hook that starts or resumes the work automatically. One web inbox becomes the control point.

The demonstrated cycle has three steps:

  1. An agent submits a plan to the gateway.
  2. Lee reviews and approves it.
  3. The approval hook resumes execution.

An infrastructure team inside the fleet built the gateway itself. Agents were now building the tools used to coordinate agents, and the arrangement worked well for Lee on one machine. Scaling the workload exposed a different set of limits.

3:233:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:23 · section reference included

Five failures under load

1. Orchestrators did the work themselves. An agent assigned to coordinate a task would start implementing it instead of dispatching it to a worker. Lee responded with a CLI harness and skills that invoke those CLIs. In his implementation, dispatch became the only available path for the orchestrator. The intended division of responsibility was enforced through the available operations.

2. Worker panes became unreadable. As managers received more tasks, they created more worker panes inside a single tmux window. Eventually, each pane was too small to inspect. Lee reports that even tmux capture-pane, his programmatic way of reading a pane, stopped returning meaningful content in this arrangement. The fleet had exhausted its display space as well as his ability to observe it; he does not describe a specific pane-layout fix.

3. Accumulated sessions exhausted memory. Lee’s Activity Monitor view shows overlapping process lists with repeated MCP entries. He describes Claude Code and MCP processes stacking up until swap was almost full and almost no free memory remained. The failure here is session accumulation consuming the host’s resources.

Slide titled Failure 3 out of memory with overlapping process-list screenshots showing memory figures and repeated MCP process entries.Open full source frame
Failure 3: out of memory, illustrated by crowded process lists.

4. Git credentials crossed workspace boundaries. The desired mapping was simple: credential A belonged to workspace A, and credential B to workspace B. Instead, credentials collided and became associated with the wrong workspaces. Lee’s fix was a clean, fully separated environment for each workspace.

5. The laptop could disappear. Lee reports losing in-progress jobs when his MacBook lost power or dropped off the network. Network disconnection is a deployment-specific part of that account: tmux itself is designed to preserve sessions across an SSH disconnect, so losing access is not inherently the same as losing running processes. In a more decisive failure, Lee loaded the laptop with so many workers that he later opened its lid to find it had restarted. The in-flight execution was gone.

His first response was a fleet recovery command:

bash

overlord boot

The command brings the fleet back up using the state already stored in files. Recovery was possible because the work was not held solely in model contexts. But restartability could not make a laptop continuously available. Lee still needed somewhere else to run the work.

4:084:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:08 · section reference included

Move execution without losing context

Lee added the two always-on Linux machines, offloading long-running coding tasks to Linux A and short-lived personal projects to Linux B. That added resources and took long-running work off the laptop while retaining one control plane. It also created a new requirement: an agent on one machine needed the context produced on another.

File-backed state made the transfer possible with familiar tools:

  1. Commit the context files on the source machine.
  2. Push the commit through Git.
  3. Connect over SSH and use tmux send-keys to tell the destination to pull.
  4. Have the destination agent read the transferred files and continue the work.

Git carries the durable context; SSH and tmux provide the signal that prompts the other machine to retrieve it.

The first version allowed two machines to change the same directory. Conflicts followed, and supposedly shared context diverged. Lee then separated machine-specific state into per-machine directories and required pull requests for shared-state changes. The separation gives local state an explicit home, while the pull-request rule puts a review boundary around changes that affect everyone.

6:076:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:07 · section reference included

Bring approvals and routing back together

Distributing execution also distributed the gateway: Lee now had an inbox on every machine. That recreated the attention problem the gateway was supposed to solve. He collapsed the inboxes into one main gateway, with every machine forwarding review requests over SSH. The gateway lives on an always-on Linux box because the sole point of control cannot depend on the MacBook staying awake.

Machine selection remained another burden. While developing across the fleet, Lee went looking for a feature and could not remember which machine he had built it on. He made Discord the single routing interface, with one bot for the MacBook, one for Linux A and one for Linux B. His phone became a remote control for the fleet. That unified the place from which he issued requests, though it did not yet eliminate the need to know where work belonged.

7:077:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:07 · section reference included

Let agents declare requirements

The distributed fleet still leaves Lee with four open engineering problems:

  • Consistency: keeping state coherent across machines.
  • Tool access: abstracting tools that remain local to the Mac, including MCP servers and the browser.
  • Credential handoff: securely moving access between instances.
  • Resource management: deciding what should run where.

He still makes placement decisions himself. The contract he wants is for an agent to declare what it needs and let the system choose where it runs.

That suggests a boundary between agent orchestration and execution infrastructure. The logical hierarchy, orchestrator and review gate sit above compute, secrets and tools; a scheduler places the workloads underneath them. Lee identifies Kubernetes as the foundation for that direction. Its scheduler places Pods using resource requirements and constraints, which fits the requirement-based placement he wants.

This is a proposed architecture, not a completed migration. Lee intends to put existing infrastructure underneath his orchestration manager and keep the custom work focused on task orchestration, review flow and context management. Kubernetes supplies useful primitives, but tool portability and secure credential handling still need engineering. In particular, Kubernetes Secrets are stored unencrypted in etcd by default; encryption at rest and restricted access require configuration.

Lee’s closing assessment is that single-machine operation works for him, while the cross-machine system remains rough and under construction. The next step is to reuse established infrastructure below the agent layer while continuing to build the coordination above it. He ends by inviting other people running agent fleets to compare notes on those unresolved operational problems.

7:538:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:53 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…