AI Engineer World's Fair 2026

Loop Engineering from first principles

Loading slides…

Read the talk

Loop Engineering from First Principles

Build coding-agent loops around measured errors, small changes and human review, using an incremental RPC-to-Effect migration to connect control theory with practical CI workflows.

From a talk by Kyle Mistele

Before you start: Familiarity with pull requests, CI workflows and coding agents is helpful; no prior knowledge of control theory or Effect is required.

What happens when nobody can review the PR?

A coding agent can keep working, verifiers can keep checking, and review agents can keep reviewing—yet the result can still be a pull request nobody wants to read. Kyle Mistele opens with an illustrative failure: six review agents surrounding a 40,000-line PR. More automated scrutiny does not, by itself, make that change understandable to the team responsible for shipping it.

Geoff Huntley’s Ralph is a useful tool for some workloads. Mistele sees its appeal especially in solo work and noncritical systems. The harder problem is using loops in a team maintaining software with customers, regulatory obligations and service-level agreements. Those constraints make the size and comprehensibility of each change part of correctness.

Slide with a cartoon character at an oven and a diagram connecting Prompt to Agent to done, with a return arrow labeled while (true).Open full source frame
Ralph Wiggum as a “software engineer,” alongside a repeating prompt-to-agent loop.
0:270:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:27 · section reference included

Loops should make code easier to read

The enthusiasm for loops began with a simple shift: instead of repeatedly prompting a coding agent yourself, design a loop that prompts it. Mistele traces the discussion from Ralph’s original post to Peter Steinberger’s advocacy of loop design. In his description of OpenClaw, loops build, review, merge and release code; other loops find bugs, including bugs in the loops themselves. He also cites Boris Cherny describing his engineering work as writing loops to prompt Claude.

Push that logic further and the volume of generated code becomes a reason to stop reading it. Verification and automated review take over, and code becomes effectively write-only. Mistele treats the accompanying claims of tenfold speed as an unsettled promise, not an established result.

His counterexamples concern both reliability and cost. Mistele says Claude Code’s terminal flicker took six months to fix, while OpenCode built a renderer in a fraction of that time; this is his historical anecdote, not a controlled development-speed comparison. The terminal-flickering issue documents the reported problem. He also points to OpenClaw stability problems and the expense of running these systems without a frontier lab’s token budget. Drawing on Matt Pocock’s argument, he adds that bad code becomes especially expensive when agents keep building on it.

Design loops that improve the code while humans continue reading it. A loop can remove bad patterns, make changes easier to review and solve difficult problems in a complex repository. The software factory can emerge incrementally, with engineering discipline applied to the loop itself.

1:552:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:55 · section reference included

Measure, correct and measure again

Control theory asks how to drive a changing system toward a desired state. For a coding loop, the system is the codebase. A sensor measures its current state, and a set point specifies the desired state. The difference is the measured error. A controller converts that error into an incremental correction, and an actuator applies the correction while other changes continue to disturb the system. Measuring again closes the loop.

Control Theory diagram showing a set point entering an error junction, followed by Controller, Actuator and System, with a disturbance input and a Sensor feeding measured output back.Open full source frame
A control loop connects measured error, controller, actuator, system and sensor.

The same basic structure appears in a thermostat: measure temperature, compare it with the setting, act, then measure again. Software engineers already encounter related mechanisms in Kubernetes autoscaling and infrastructure-as-code reconciliation. Mistele also describes PostgreSQL autovacuum and React’s virtual DOM as using or approximating control loops. The common requirements are a system you can change, a problem you can measure and feedback about the result.

Incremental corrections reduce the chance of oversteering. Instead of attempting the entire desired end state in one change, the controller makes a bounded move and checks its effect. That is the useful distinction between a feedback loop and a blind loop. It is not a categorical rejection of Ralph: Mistele explicitly recognizes that the best Ralph implementations already apply control principles and that Ralph is a teaching device, not an instruction to abandon judgment.

4:334:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:33 · section reference included

Separate the responsibilities, even when one agent performs them

Start an agentic control loop by defining a property the codebase should satisfy, then choose how to detect deviations. Simply repeating a Bash command does not guarantee incremental progress; the sensing and selection policy must supply that discipline.

Sensor approachSuitable ingredients
DeterministicESLint rules, ast-grep, Packwerk
AgenticAn agent, a skill and natural-language rules
HybridA pipeline combining deterministic checks and agent judgment

These are alternatives for measuring the discrepancy, not requirements to make every part of the loop agentic.

The implementation boundaries can overlap. Mistele uses Aiden Bai’s React Doctor as an example of a hybrid sensor and controller: it identifies React problems, prioritizes fixes and explains how to make them. Likewise, one agent can select a change and implement it in the same context window. Combining components does not remove the controller’s responsibility to limit the size of a change and choose the right direction. Repeated oversized or incorrect corrections can quickly destabilize the codebase.

6:597:15
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:59 · section reference included

Choose a measurable maintenance workload

Pattern cleanup is only one application. A loop could check an API against someone else’s OpenAPI specification, maintain compliance with the particular MCP specification version a server targets, mirror a project between Python and TypeScript, or keep a Vite-based Next.js fork aligned with upstream. Each candidate needs the same three properties: a measurable discrepancy, incremental corrections and feedback on their quality.

HumanLayer’s example is an incremental migration of its RPC API to Effect. The team first adopted Effect for race-prone code, liked the result and began extending it across the codebase. In the recording, Mistele places a small procedure beside its Effect rewrite to establish the transformation the loop will perform. The syntax is incidental to the loop design; this is a migration chosen for the team’s needs, not a recommendation that every codebase adopt Effect.

8:298:40
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:29 · section reference included

Find old procedures—and stop adding new ones

The first implementation task is a sensor that finds unmigrated RPC procedures. An agent or a text search could do this, but Mistele chooses ast-grep for structural matching. Its scanning rules can live outside the ordinary TypeScript and ESLint configuration, avoiding reliance on checks an agent might suppress with inline comments. Additional rules can target other unwanted patterns, with granular include and exclude paths across supported languages in a monorepo.

The raw results need normalization before they become a useful control input. In Mistele’s workflow, ast-grep produced roughly 50 keys per violation; the team retained four fields and sorted the results deterministically. That count describes this workflow, not a fixed output contract. The practical purpose of processing ast-grep’s JSON output is to turn a verbose scan into a stable list that can be compared between revisions.

Before migrating the backlog, prevent it from growing:

  1. Run a full scan on main.
  2. Sort the violations deterministically and commit the baseline to version control.
  3. Scan each pull request and check whether it introduces any new unmigrated procedures.
  4. Require new procedures to use Effect while existing procedures are migrated incrementally.

This lets the repository carry known migration debt without treating new debt as acceptable.

Other developers are still changing the system while the loop works. Without the regression check, their changes could undo its progress. Mistele loosely calls the baseline check a disturbance dampener: it is a practical safeguard around the control loop rather than a perfect one-to-one mapping to a control-theory component.

9:409:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:40 · section reference included

Choose a small correction with a useful purpose

A minimal controller can use Bash and jq to select the first violation from the sorted list. A more selective controller can use structural analysis to choose the smallest unmigrated procedure, limiting the scope of the next change. An agent could make the selection, or select and implement together, but Mistele recommends keeping deterministic decisions in deterministic code.

The migration also has an operational purpose: better error handling and instrumentation. A more ambitious controller could use telemetry to prioritize procedures with frequent errors, weak instrumentation or gaps in application performance monitoring. The control signal would then contain both the selected procedure and the evidence explaining what should improve. That gives the actuator a reason to improve the implementation rather than mechanically translate its syntax. The accompanying diagram makes the handoff concrete: Sentry traces and errors inform selection, and sample trace data travels with the chosen procedure to the agent.

Data-Driven Controller diagram with AST-Grep Rules, steps to fetch Sentry traces and errors by procedure and pick the most errors or worst performance, and an Actuator (Agent) receiving the selected procedure and sample trace data.Open full source frame
A data-driven controller selects a procedure using Sentry traces and errors, then passes trace data to the actuator.
11:2111:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:21 · section reference included

Give the actuator local patterns to follow

The actuator is a CLI coding agent plus a skill. The skill deserves substantial attention, but it should evolve through observed results rather than being treated as a specification that must be perfected before the first run. HumanLayer writes golden patterns by hand: idiomatic examples that show the agent how this repository expects a change to look. Without those examples, the agent falls back on documentation and patterns learned elsewhere.

Pass the skill and the controller’s selected target into the actuator together. Include a response template in the skill so the agent’s final message can serve as the pull-request description. Once the agent finishes, deterministic workflow steps commit the change, push the branch and create the PR. The agent handles implementation; ordinary automation handles publication of its proposed change.

12:2612:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:26 · section reference included

Use existing CI, then fix the correction friction

Existing CI is a practical place to run the loop. GitHub Actions, GitLab or CircleCI already provides repository access, secrets, dispatch and scheduling. A workflow can perform one sense–control–actuate iteration, create a PR and run on a daily schedule. There is no need for a separate cluster merely to orchestrate this sequence. The intended result is a small change waiting for review each morning.

That initial design was frustrating enough that HumanLayer turned it off. Correcting an output meant checking out the branch, changing the skill, changing the code, committing and pushing again. The migration itself might be small, but steering the automation demanded too much manual work. A useful loop therefore needs a low-friction way for a human to correct both the current proposal and the instructions that will shape future proposals.

13:2213:36
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:22 · section reference included

Turn review comments into persistent steering

Add a Markdown feedback file to version control and load it deterministically into the actuator’s context on every run, after the controller has selected the work. Then label every generated PR with an identifier for its loop. The label matters when several workflows operate in the same repository: each must recognize its own PRs and respond only to feedback intended for it.

The correction path starts with a reviewer leaving an /iterate comment:

  1. The corresponding loop workflow receives the comment trigger.
  2. It loads the PR diff, comments, review comments and description into the agent’s context alongside the skill.
  3. It instructs the agent to fix the proposed code and update the feedback file.

The review now produces both a repair to the current PR and reusable guidance for subsequent actuator runs.

Keeping that guidance in version control makes changes to the loop’s behavior inspectable. The team can see how its instructions evolved and revert them when necessary. Human steering becomes part of the repository’s reviewable history rather than a series of manual skill edits that are difficult to reconstruct.

14:1314:24
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:13 · section reference included

Stop producing changes when review is blocked

A scheduled loop can outpace its reviewers even when each individual change is small. HumanLayer encountered this during travel, customer visits and other work: PRs accumulated, duplicated effort and conflicted. Useful maintenance became another backlog demanding attention.

The existing PR label supplies a simple backpressure mechanism. Before checking out code, installing dependencies or invoking an agent, query for open PRs carrying the loop’s label. If one exists, stop the new run. The gate permits at most one open PR per loop.

An outstanding PR is a conservative signal that the previous output has not cleared the human-review boundary. Producing another change does not resolve that bottleneck. Waiting prevents a stack of new proposals from building up behind it, reduces duplicate work and, as Mistele qualifies it, hopefully avoids conflicts.

15:1015:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:10 · section reference included

Increase throughput as confidence and review capacity grow

Once the loop produces changes the team trusts, the next question is how to accelerate it. Mistele cites a backlog of 150 RPC procedures and estimates roughly six months at the preceding pace of one procedure per daily run. That is a planning estimate, not a measured completion rate; review delays also affect how quickly the backlog clears.

He proposes two ways to increase the work selected in an iteration:

  • Larger batches: Have the controller select three or five procedures instead of one.
  • Separate implementation contexts: Select a batch, then migrate each procedure in its own implementation phase and context window. Mistele expects this separation to be cheaper and more reliable; he presents no measured comparison.

The distinction is between how much work the controller chooses and how much context each implementation receives.

Another proposed extension distributes review ownership: run the workflow four times and give one PR to each of four teammates. This would require adjusting the earlier single-open-PR policy; the talk does not specify that concurrency design. The underlying constraint remains human capacity to read the resulting changes.

The resulting operating model improves the code incrementally, slows down when review cannot keep up and accepts corrections through the same PR interface the team already uses. Mistele closes by inviting viewers to try the offered skill and share what they build. The useful endpoint is a loop whose output remains readable and whose behavior the team can steer as it learns.

Closing slide with a skill installation command above five bullets: Build control loops; Read the code; Make the code better; Manage risk with incremental change; Use adaptive flow control & steering.Open full source frame
Closing recommendations pair control loops and code review with incremental change, adaptive flow control and steering.
16:1716:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:17 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…