AI Engineer World's Fair 2025
12-Factor Agents: Patterns of reliable LLM applications
Loading slides…
Read the talk
12-Factor Agents: Patterns of Reliable LLM Applications
Dex Horthy reduces an agent to structured model output, deterministic execution, explicit state, and a controlled loop—then shows where small doses of model judgment outperform a free-running agent.
From a talk by Dex Horthy
At a glance
Ideas worth remembering
Use an agent only where model-driven judgment adds value. If the correct procedure is already an exact sequence, deterministic code may be faster to build and easier to trust.
Treat tool use as structured model output followed by application-controlled execution. Own the dispatcher, loop, branching, and termination rules.
Persist execution and business state outside the model so long-running workflows can pause, receive callbacks, and resume through ordinary APIs.
Context engineering means selecting and representing prompts, history, memory, retrieval, results, and errors—not filling the largest available context window.
Embed focused three-to-ten-step agent loops inside deterministic workflows, with explicit human clarification, correction, and approval paths.
Prefer tooling that generates inspectable, owned scaffolding and removes peripheral work without hiding prompts, context, state, or control flow.
The prototype works—until reliability matters
Agent frameworks can get a project to an exciting 70–80% quality bar quickly. The trouble begins when the remaining failures matter to customers. Horthy describes tracing through seven layers of a call stack just to discover how a prompt was assembled or how tools reached the model. At that point, the abstraction that accelerated the prototype can obstruct the work required to make it dependable.
His early DevOps agent supplied the cleanest warning. He gave it a Makefile and permission to run Make commands, but it executed the build steps in the wrong order. Two hours of prompt editing eventually specified the exact sequence. The resulting agent could follow the procedure, but the same procedure could have been a Bash script written in about 90 seconds. Once judgment has been designed out of a task, deterministic code is usually the simpler implementation.
Conversations with more than 100 founders, builders, and engineers led Horthy to a less theatrical model of production agents: they are mostly ordinary software with LLM decisions inserted where language understanding or flexible judgment earns its cost. Teams were not necessarily replacing their applications with new agent stacks. They were adding small patterns to existing code.
The resulting 12-Factor Agents guide is therefore neither a demand for greenfield rewrites nor an attack on frameworks. Horthy treats it as a wish list for tools that preserve development speed while exposing the prompts, state, context, and control flow builders need to improve reliability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A tool call is JSON followed by ordinary code
The first useful primitive is natural language converted into structured data. A model receives a sentence and emits JSON describing an intended action. That conversion is separate from execution: the application still decides how to validate, dispatch, and run the requested operation.
This is the point behind Horthy’s deliberately provocative claim that “tool use” can be a harmful abstraction. He is not objecting to agents acting on external systems. He is objecting to language that makes the mechanism seem magical. The model emits JSON; deterministic code interprets it using a loop, switch statement, API client, or other familiar construct; the application may then return the result to the model.
The naive agent loop is compact: receive an event, ask the model for the next step, execute it, append the result to context, and repeat until the model says the task is done. The sequence generated during one run becomes a materialized graph, but the developer did not define that graph in advance.
Longer workflows expose the cost of that simplicity. Every result expands the context, and merely having room for a very large prompt does not guarantee a tight or reliable decision. Horthy’s claim is practical rather than benchmarked here: limiting and controlling the tokens sent to the model generally produces better results than indiscriminately accumulating history.
A useful working definition follows. An agent consists of four owned parts:
- Prompt: instructions for choosing the next step.
- Dispatcher: deterministic code that interprets the model’s structured output.
- Context builder: the logic that selects and represents what has happened.
- Loop controller: the rules for continuing, branching, summarizing, switching strategies, or stopping.
Owning these pieces lets the application add explicit breaks, branches, summaries, and model-based checks instead of accepting an opaque loop.
The agent loop without the mystique
Read the diagram as text
- Event or current state. A user message, outage, or prior tool result supplies the current task state.
- Model chooses next step. The prompt asks the model to emit a structured action or indicate completion.
- Deterministic dispatcher. A switch statement or equivalent code interprets the JSON.
- Curated context. The application records the useful result and determines what the next model call should see.
- Event or current state → Model chooses next step: Build prompt.
- Model chooses next step → Deterministic dispatcher: Structured output.
- Deterministic dispatcher → Curated context: Execution result.
- Curated context → Model chooses next step: Continue if needed.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Persist state so long-running work can disappear and return
Once the loop is explicit, execution state and business state can live together in application-owned storage. Execution state includes the current step, next step, and retry counts. Business state includes messages, user-visible data, and approvals the workflow is waiting to receive. Keeping both available makes the agent’s progress inspectable in the same terms as the product experience.
That state also enables ordinary launch, pause, and resume APIs. Put the agent behind a REST API or MCP server. When it starts a long-running operation, interrupt the loop and serialize its context into a database. The external operation eventually calls back with a state ID and result. The application reloads the stored state, appends the result, and invokes the model again.
The model does not need to remain alive or remember that asynchronous work occurred. From its perspective, the next invocation simply contains the prior state plus a new result. Durability belongs to the application, not to the model process.
Pause and resume a long-running agent operation
Read the diagram as text
- Incoming request. A REST or MCP request launches the workflow.
- Agent requests long-running work. The model selects an operation that will finish asynchronously.
- Persist context and state ID. The application serializes owned workflow state into a database.
- Receive result callback. The operation returns its result with the corresponding state ID.
- Restore and resume. The application reloads state, appends the result, and calls the model again.
- Incoming request → Agent requests long-running work: Load context.
- Agent requests long-running work → Persist context and state ID: Interrupt loop.
- Persist context and state ID → Receive result callback: Wait outside model process.
- Receive result callback → Restore and resume: Look up by state ID.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reliability is largely a context-construction problem
Prompt-generating abstractions can provide an excellent starting point, but Horthy expects teams pushing beyond an initial quality threshold to inspect and edit the prompt directly. His argument is about experimental control: if model output depends on the input tokens, builders need enough access to vary those tokens, compare alternatives, and evaluate which construction works best.
Context deserves the same ownership. A standard messages array is one representation, not a law. An application can model events, conversation history, retrieved material, memory, and business state however it wants, then serialize the relevant information into one or more model messages. The goal is to communicate what happened so far with high density and clarity.
This folds several commonly separated mechanisms into one engineering problem:
- Prompt instructions define the decision the model should make.
- History records relevant prior actions and outcomes.
- Memory carries selected information across interactions.
- Retrieval inserts external material needed for the present decision.
All four ultimately become tokens in the current model input. The architecture may distinguish them for storage and maintenance, but the model receives a constructed context.
Errors show why indiscriminate history is dangerous. Returning a failed tool call and its error can help the model repair an invalid API request or respond to a temporary outage. Continually appending failures, however, can trap the loop in stale error context. Horthy recommends clearing pending errors after a valid action, summarizing prior failures when useful, and omitting entire stack traces unless their details actually help select the next step.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put people in the control flow, then keep autonomy small
Human contact should be a first-class action rather than an exceptional escape hatch. Horthy recommends giving the model distinct intents such as completion, clarification, or escalation. Expressing that intent in natural language also moves an important choice to the beginning of generation, where the model decides what kind of interaction it needs before producing the rest of the action.
The interface should meet people where they already work. Email, Slack, Discord, and SMS can carry approvals and corrections without forcing every user to keep another ChatGPT-style tab open. This does not make channel delivery trivial, but it treats human participation as part of the workflow rather than a handoff outside it.
The architectural companion is the small, focused agent: a loop of roughly three to ten steps embedded inside a mostly deterministic directed graph. Conventional code handles the predictable pipeline. The model handles a bounded region where natural-language interpretation, reprioritization, or recovery is useful.
HumanLayer’s deployment bot illustrates the pattern. Deterministic CI/CD code waits until a GitHub pull request is merged and development tests pass. A focused deployment agent then proposes the next action. If it proposes the front end first, a human can respond that the back end should go first. The model converts that correction into structured workflow state, proposes the back-end deployment, waits for approval, executes it, and then returns to the front end.
After successful deployment, deterministic code resumes and runs end-to-end tests against production. If deployment fails, a separate rollback agent can take over its own bounded responsibility. Horthy characterizes the resulting system as manageable even when the overall product has many tools and steps, because each agent receives a limited context and a clear job.
More capable models may gradually absorb larger parts of these pipelines. Horthy expects the boundary to move: start with deterministic software, insert LLM decisions, and expand their scope as reliability improves. The durable skill is learning to operate near the boundary of what a model can do consistently, then using prompts, context, state, control flow, and human judgment to make that difficult capability dependable.
A focused deployment agent inside deterministic CI/CD
Read the diagram as text
- PR merged and development tests pass. deterministic — Deterministic CI/CD reaches the deployment decision point.
- Agent proposes front-end deployment. agent — The model converts the deployment goal into the next structured action.
- Human requests back end first. human — A person changes the proposed order in natural language.
- Approve and deploy back end. agent — The agent proposes the corrected next step and the workflow executes it.
- Deploy front end. agent — The agent returns to the remaining deployment task.
- Run production end-to-end tests. deterministic — Control returns to deterministic code after successful deployment.
- PR merged and development tests pass → Agent proposes front-end deployment: Enter focused agent loop.
- Agent proposes front-end deployment → Human requests back end first: Request human review.
- Human requests back end first → Approve and deploy back end: Natural language to structured action.
- Approve and deploy back end → Deploy front end: Continue remaining work.
- Deploy front end → Run production end-to-end tests: Exit agent loop.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the model stateless and make the scaffolding yours
Horthy’s last factor describes the agent as effectively stateless: the application owns and supplies the state required for each step. He notes that the reducer terminology is imperfect for a multistep process, but the implementation lesson survives the naming dispute. Durable memory, workflow progress, and business records should not depend on an in-memory model session.
That preference shapes his desired tooling. Rather than a wrapper around hidden internal behavior, he wants generated scaffolding that developers can inspect, modify, and keep. The team was working on Create Twelve-Factor Agent with this model: generate an understandable starting structure, then let the developer own the resulting code.
The ending sharpens the division of labor. Supporting tools should remove peripheral operational work, leaving builders time for the difficult AI-specific decisions: prompt construction, context selection, control flow, and human collaboration. A framework that hides those decisions may remove precisely the controls needed to cross the production reliability gap.
Horthy also points to work on an A2H Protocol intended to consolidate how agents contact humans. That proposal remains only briefly described in the recording. The supported design principle is narrower: people improve agents when clarification, approval, and intervention are available through explicit actions and familiar communication channels.
The practical conclusion is intentionally ordinary. Use a model where converting language into a structured decision creates value. Keep state and execution in software you control. Build the smallest useful loop, let people participate, and expand autonomy only when the surrounding system can preserve reliability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The public guide containing all twelve factors, examples, and project materials discussed in the talk.
- Dex Horthy on XReference
The speaker profile supplied with the recording.
Further reading
- 12-Factor Agents talk pageReference
The AI Engineer page for the recording, with chapters and a timestamped reading version.
Related talks
- How We Build Effective Agents
Barry Zhang likewise argues for avoiding agents when simpler workflows suffice and for keeping agent architectures simple.
- Agents vs Workflows: Why Not Both?
Sam Bhagwat examines the same architectural boundary between model-directed loops and structured workflows.
- No Vibes Allowed: Solving Hard Problems in Complex Codebases
Dex Horthy develops the context-engineering side of this talk into a fuller workflow for managing difficult coding tasks.
Full transcript
Loading transcript…