AI Engineer World's Fair 2025

12-Factor Agents: Patterns of reliable LLM applications

Loading slides…

Read the talk

12-Factor Agents: Patterns of Reliable LLM Applications

Dex Horthy reduces an agent to structured model output, deterministic execution, explicit state, and a controlled loop—then shows where small doses of model judgment outperform a free-running agent.

From a talk by Dex Horthy

At a glance

Ideas worth remembering

  • Use an agent only where model-driven judgment adds value. If the correct procedure is already an exact sequence, deterministic code may be faster to build and easier to trust.

  • Treat tool use as structured model output followed by application-controlled execution. Own the dispatcher, loop, branching, and termination rules.

  • Persist execution and business state outside the model so long-running workflows can pause, receive callbacks, and resume through ordinary APIs.

  • Context engineering means selecting and representing prompts, history, memory, retrieval, results, and errors—not filling the largest available context window.

  • Embed focused three-to-ten-step agent loops inside deterministic workflows, with explicit human clarification, correction, and approval paths.

  • Prefer tooling that generates inspectable, owned scaffolding and removes peripheral work without hiding prompts, context, state, or control flow.

The prototype works—until reliability matters

Agent frameworks can get a project to an exciting 70–80% quality bar quickly. The trouble begins when the remaining failures matter to customers. Horthy describes tracing through seven layers of a call stack just to discover how a prompt was assembled or how tools reached the model. At that point, the abstraction that accelerated the prototype can obstruct the work required to make it dependable.

Captures the Makefile example as Horthy contrasts two hours of agent prompting with a Bash script that could be written in roughly 90 seconds.Open full source frame
Captures the Makefile example as Horthy contrasts two hours of agent prompting with a Bash script that could be written in roughly 90 seconds.

His early DevOps agent supplied the cleanest warning. He gave it a Makefile and permission to run Make commands, but it executed the build steps in the wrong order. Two hours of prompt editing eventually specified the exact sequence. The resulting agent could follow the procedure, but the same procedure could have been a Bash script written in about 90 seconds. Once judgment has been designed out of a task, deterministic code is usually the simpler implementation.

Conversations with more than 100 founders, builders, and engineers led Horthy to a less theatrical model of production agents: they are mostly ordinary software with LLM decisions inserted where language understanding or flexible judgment earns its cost. Teams were not necessarily replacing their applications with new agent stacks. They were adding small patterns to existing code.

The resulting 12-Factor Agents guide is therefore neither a demand for greenfield rewrites nor an attack on frameworks. Horthy treats it as a wish list for tools that preserve development speed while exposing the prompts, state, context, and control flow builders need to improve reliability.

0:150:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:15 · section reference included

A tool call is JSON followed by ordinary code

The first useful primitive is natural language converted into structured data. A model receives a sentence and emits JSON describing an intended action. That conversion is separate from execution: the application still decides how to validate, dispatch, and run the requested operation.

Illustrates the naive agent loop: event, model-selected action, result appended to context, and repetition until completion.Open full source frame
Illustrates the naive agent loop: event, model-selected action, result appended to context, and repetition until completion.

This is the point behind Horthy’s deliberately provocative claim that “tool use” can be a harmful abstraction. He is not objecting to agents acting on external systems. He is objecting to language that makes the mechanism seem magical. The model emits JSON; deterministic code interprets it using a loop, switch statement, API client, or other familiar construct; the application may then return the result to the model.

The naive agent loop is compact: receive an event, ask the model for the next step, execute it, append the result to context, and repeat until the model says the task is done. The sequence generated during one run becomes a materialized graph, but the developer did not define that graph in advance.

Longer workflows expose the cost of that simplicity. Every result expands the context, and merely having room for a very large prompt does not guarantee a tight or reliable decision. Horthy’s claim is practical rather than benchmarked here: limiting and controlling the tokens sent to the model generally produces better results than indiscriminately accumulating history.

A useful working definition follows. An agent consists of four owned parts:

  • Prompt: instructions for choosing the next step.
  • Dispatcher: deterministic code that interprets the model’s structured output.
  • Context builder: the logic that selects and represents what has happened.
  • Loop controller: the rules for continuing, branching, summarizing, switching strategies, or stopping.

Owning these pieces lets the application add explicit breaks, branches, summaries, and model-based checks instead of accepting an opaque loop.

The agent loop without the mystique

Event or current stateA user message, outage, or prior tool result supplies the current task state.
Model chooses next stepThe prompt asks the model to emit a structured action or indicate completion.
Deterministic dispatcherA switch statement or equivalent code interprets the JSON.
Curated contextThe application records the useful result and determines what the next model call should see.
The model proposes a structured next action. Application code executes it, curates the resulting context, and decides whether another iteration should run.
Read the diagram as text
  • Event or current state. A user message, outage, or prior tool result supplies the current task state.
  • Model chooses next step. The prompt asks the model to emit a structured action or indicate completion.
  • Deterministic dispatcher. A switch statement or equivalent code interprets the JSON.
  • Curated context. The application records the useful result and determines what the next model call should see.
  • Event or current state → Model chooses next step: Build prompt.
  • Model chooses next step → Deterministic dispatcher: Structured output.
  • Deterministic dispatcher → Curated context: Execution result.
  • Curated context → Model chooses next step: Continue if needed.
3:444:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:44 · section reference included

Persist state so long-running work can disappear and return

Once the loop is explicit, execution state and business state can live together in application-owned storage. Execution state includes the current step, next step, and retry counts. Business state includes messages, user-visible data, and approvals the workflow is waiting to receive. Keeping both available makes the agent’s progress inspectable in the same terms as the product experience.

Useful state-model frame distinguishing execution state—steps and retries—from business state such as messages, displayed data, and approvals.Open full source frame
Useful state-model frame distinguishing execution state—steps and retries—from business state such as messages, displayed data, and approvals.

That state also enables ordinary launch, pause, and resume APIs. Put the agent behind a REST API or MCP server. When it starts a long-running operation, interrupt the loop and serialize its context into a database. The external operation eventually calls back with a state ID and result. The application reloads the stored state, appends the result, and invokes the model again.

The model does not need to remain alive or remember that asynchronous work occurred. From its perspective, the next invocation simply contains the prior state plus a new result. Durability belongs to the application, not to the model process.

Pause and resume a long-running agent operation

Incoming requestA REST or MCP request launches the workflow.
Agent requests long-running workThe model selects an operation that will finish asynchronously.
Persist context and state IDThe application serializes owned workflow state into a database.
Receive result callbackThe operation returns its result with the corresponding state ID.
Restore and resumeThe application reloads state, appends the result, and calls the model again.
Application-owned state lets the model loop stop while external work runs, then resume from a callback without relying on model memory.
Read the diagram as text
  • Incoming request. A REST or MCP request launches the workflow.
  • Agent requests long-running work. The model selects an operation that will finish asynchronously.
  • Persist context and state ID. The application serializes owned workflow state into a database.
  • Receive result callback. The operation returns its result with the corresponding state ID.
  • Restore and resume. The application reloads state, appends the result, and calls the model again.
  • Incoming request → Agent requests long-running work: Load context.
  • Agent requests long-running work → Persist context and state ID: Interrupt loop.
  • Persist context and state ID → Receive result callback: Wait outside model process.
  • Receive result callback → Restore and resume: Look up by state ID.
7:147:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:14 · section reference included

Reliability is largely a context-construction problem

Prompt-generating abstractions can provide an excellent starting point, but Horthy expects teams pushing beyond an initial quality threshold to inspect and edit the prompt directly. His argument is about experimental control: if model output depends on the input tokens, builders need enough access to vary those tokens, compare alternatives, and evaluate which construction works best.

Provides the concrete error-handling lesson: do not blindly accumulate failures or full stack traces in the context window.Open full source frame
Provides the concrete error-handling lesson: do not blindly accumulate failures or full stack traces in the context window.

Context deserves the same ownership. A standard messages array is one representation, not a law. An application can model events, conversation history, retrieved material, memory, and business state however it wants, then serialize the relevant information into one or more model messages. The goal is to communicate what happened so far with high density and clarity.

This folds several commonly separated mechanisms into one engineering problem:

  • Prompt instructions define the decision the model should make.
  • History records relevant prior actions and outcomes.
  • Memory carries selected information across interactions.
  • Retrieval inserts external material needed for the present decision.

All four ultimately become tokens in the current model input. The architecture may distinguish them for storage and maintenance, but the model receives a constructed context.

Errors show why indiscriminate history is dangerous. Returning a failed tool call and its error can help the model repair an invalid API request or respond to a temporary outage. Continually appending failures, however, can trap the loop in stale error context. Horthy recommends clearing pending errors after a valid action, summarizing prior failures when useful, and omitting entire stack traces unless their details actually help select the next step.

8:148:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:14 · section reference included

Put people in the control flow, then keep autonomy small

Human contact should be a first-class action rather than an exceptional escape hatch. Horthy recommends giving the model distinct intents such as completion, clarification, or escalation. Expressing that intent in natural language also moves an important choice to the beginning of generation, where the model decides what kind of interaction it needs before producing the rest of the action.

Shows the deployment example returning to deterministic production tests or handing failure to a separate rollback agent.Open full source frame
Shows the deployment example returning to deterministic production tests or handing failure to a separate rollback agent.

The interface should meet people where they already work. Email, Slack, Discord, and SMS can carry approvals and corrections without forcing every user to keep another ChatGPT-style tab open. This does not make channel delivery trivial, but it treats human participation as part of the workflow rather than a handoff outside it.

The architectural companion is the small, focused agent: a loop of roughly three to ten steps embedded inside a mostly deterministic directed graph. Conventional code handles the predictable pipeline. The model handles a bounded region where natural-language interpretation, reprioritization, or recovery is useful.

HumanLayer’s deployment bot illustrates the pattern. Deterministic CI/CD code waits until a GitHub pull request is merged and development tests pass. A focused deployment agent then proposes the next action. If it proposes the front end first, a human can respond that the back end should go first. The model converts that correction into structured workflow state, proposes the back-end deployment, waits for approval, executes it, and then returns to the front end.

After successful deployment, deterministic code resumes and runs end-to-end tests against production. If deployment fails, a separate rollback agent can take over its own bounded responsibility. Horthy characterizes the resulting system as manageable even when the overall product has many tools and steps, because each agent receives a limited context and a clear job.

More capable models may gradually absorb larger parts of these pipelines. Horthy expects the boundary to move: start with deterministic software, insert LLM decisions, and expand their scope as reliability improves. The durable skill is learning to operate near the boundary of what a model can do consistently, then using prompts, context, state, control flow, and human judgment to make that difficult capability dependable.

A focused deployment agent inside deterministic CI/CD

PR merged and development tests passdeterministic — Deterministic CI/CD reaches the deployment decision point.
Agent proposes front-end deploymentagent — The model converts the deployment goal into the next structured action.
Human requests back end firsthuman — A person changes the proposed order in natural language.
Approve and deploy back endagent — The agent proposes the corrected next step and the workflow executes it.
Deploy front endagent — The agent returns to the remaining deployment task.
Run production end-to-end testsdeterministic — Control returns to deterministic code after successful deployment.
The model handles a bounded sequencing decision and accepts human correction; conventional code owns the surrounding pipeline and production tests.
Read the diagram as text
  • PR merged and development tests pass. deterministic — Deterministic CI/CD reaches the deployment decision point.
  • Agent proposes front-end deployment. agent — The model converts the deployment goal into the next structured action.
  • Human requests back end first. human — A person changes the proposed order in natural language.
  • Approve and deploy back end. agent — The agent proposes the corrected next step and the workflow executes it.
  • Deploy front end. agent — The agent returns to the remaining deployment task.
  • Run production end-to-end tests. deterministic — Control returns to deterministic code after successful deployment.
  • PR merged and development tests pass → Agent proposes front-end deployment: Enter focused agent loop.
  • Agent proposes front-end deployment → Human requests back end first: Request human review.
  • Human requests back end first → Approve and deploy back end: Natural language to structured action.
  • Approve and deploy back end → Deploy front end: Continue remaining work.
  • Deploy front end → Run production end-to-end tests: Exit agent loop.
11:1311:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:13 · section reference included

Keep the model stateless and make the scaffolding yours

Horthy’s last factor describes the agent as effectively stateless: the application owns and supplies the state required for each step. He notes that the reducer terminology is imperfect for a multistep process, but the implementation lesson survives the naming dispute. Durable memory, workflow progress, and business records should not depend on an in-memory model session.

That preference shapes his desired tooling. Rather than a wrapper around hidden internal behavior, he wants generated scaffolding that developers can inspect, modify, and keep. The team was working on Create Twelve-Factor Agent with this model: generate an understandable starting structure, then let the developer own the resulting code.

The ending sharpens the division of labor. Supporting tools should remove peripheral operational work, leaving builders time for the difficult AI-specific decisions: prompt construction, context selection, control flow, and human collaboration. A framework that hides those decisions may remove precisely the controls needed to cross the production reliability gap.

Horthy also points to work on an A2H Protocol intended to consolidate how agents contact humans. That proposal remains only briefly described in the recording. The supported design principle is narrower: people improve agents when clarification, approval, and intervention are available through explicit actions and familiar communication channels.

The practical conclusion is intentionally ordinary. Use a model where converting language into a structured decision creates value. Keep state and execution in software you control. Build the smallest useful loop, let people participate, and expand autonomy only when the surrounding system can preserve reliability.

14:1314:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:13 · section reference included

Resources

From the talk

  • The public guide containing all twelve factors, examples, and project materials discussed in the talk.

  • The speaker profile supplied with the recording.

Transcript download
Full transcript

Loading transcript…