AI Engineer World's Fair 2026
Build Systems, Not Code
Loading slides…
Read the talk
Build Systems, Not Code
A house-hunting agent shows where software engineering moves when models write the code: workflows, component boundaries, durable memory, safe retries, and human authority.
From a talk by Angie Jones
Finding the pleasure of building again
One Friday night, Angie Jones was building agents for operational tasks when the sun set. Dinner came and went. She had fallen back into a familiar developer flow: absorbed in making something work, with the thrill of building intact. That experience answered a worry she hears from developers using coding agents—that automation is taking the enjoyable parts of programming and leaving the tedious work behind.
The opportunity is one layer above generating code. Building an agent means designing the system in which it operates. The building blocks change, but decisions about responsibilities, dependencies, state, and failure still require engineering judgment. Relocation Scout, Jones’s house-hunting example, makes those decisions concrete.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A house search outlasts a prompt
Point an agent at some property listings and ask it to rank them, and a single prompt may do the job. But a house search rarely finishes in a day. Tomorrow’s search needs access to what was learned today, even if tomorrow begins with an empty conversation. Relocation Scout therefore needs knowledge persisted outside the session, where it can be reloaded or queried in a fresh context.
The agent is only one component. Its environment includes files, tools, people, and potentially other agents. Relocation Scout pulls in listings and neighborhood signals, weighs them against the buyer’s preferences, and returns a ranked shortlist. Those inputs and outputs establish its place in a larger system.
Before asking a coding agent to implement it, define its job, dependencies, boundaries, and failure behavior. What does it need to do its work? What happens when a dependency breaks? Which responsibilities belong elsewhere? A coding agent can help build the component, but these questions determine what component should be built.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give the goal a path
Traditional software already makes workflows explicit in CI/CD pipelines and ticket life cycles. Agentic systems need the same treatment. A /goal command can express a desired outcome, but reviewing a listing still requires a path through the work.
For Relocation Scout, that path is:
- Gather the information needed to review the listing.
- Evaluate it against the buyer’s criteria.
- Decide what to do, act, and record the result.
- Stop, retry, or escalate.
The workflow determines what context each step needs, which work the agent handles directly, and where a tool or person takes over. Those handoffs are architectural decisions, not details to leave implicit in the goal.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Find the jobs hiding in the prompt
An oversized class, a sprawling function, or a service with too many responsibilities is a familiar code smell. The agent equivalent is a giant prompt. It starts reasonably: instructions for sizing up a property. Then an edge case needs a note. A safety rule gets added. Another important exception follows. Eventually, the instruction file is trying to do everything.
Relocation Scout’s prompt contains four distinct jobs: pulling and normalizing listings, formatting the shortlist, calculating commutes, and researching neighborhoods. Jones connects that overlong script with agent drift. The reason to pull the jobs apart is practical: each becomes easier to reason about, test, and change. Creating more pieces is not itself the objective.
Decomposition identifies separate responsibilities; separation of concerns puts them in appropriate places. This is the same kind of decision as choosing between a controller and a service layer, or separating business logic from presentation. Relocation Scout simply offers a different set of destinations.
| Responsibility | Component |
|---|---|
| Normalize listings | Reusable skill |
| Format shortlist entries consistently | Output schema |
| Calculate commute time | Script |
| Research neighborhoods | Sub-agent |
The normalization process becomes a capability the agent can load. The schema defines the shape of the result. A script handles the calculation, while the substantial research task gets its own agent scope. Each responsibility now has a clear home.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reuse the capability, not every instruction
Suppose the house search expands to three cities. Each market can load the same listing-normalization skill. The process is written once, reused across agents, and potentially shared with other people, much as a library or package would be.
A sub-agent provides another kind of module. Architecturally, Jones compares it to a function: give it one specific task and call it when that task is needed. The neighborhood researcher can focus on its assignment without carrying the entire parent session’s context. That focused responsibility makes it useful across different markets and workflows.
Reuse still needs judgment. Some instructions are local to one workflow, and turning them into an abstraction can cost more than it saves. Modularity should make a capability easier to use and maintain, not force every instruction into a shared component.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Assign determinism, judgment, and authority
An agent’s ability to attempt a task does not make it the best place to perform that task. Commute calculations and filtering out previously seen listings belong in ordinary code. Interpreting messy neighborhood information and judging which listings deserve attention are better uses of model reasoning.
Giving every step to the model introduces unnecessary variability: the same kind of task may produce different output on different days. Jones’s rule of thumb is to reach for code when a task has an exact answer and use an agent when it needs interpretation. Keeping deterministic work in code can also avoid unnecessary model expense and complexity.
| Actor | Responsibility | Relocation Scout example |
|---|---|---|
| Code | Determinism | Calculate commute; remove seen listings |
| Agent | Judgment | Select listings worth a closer look |
| Human | Authority | Approve booking a tour |
The final distinction matters: evaluating a property and authorizing an action are separate responsibilities. A promising recommendation does not, by itself, grant permission to book the visit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn an assessment into a durable contract
Freeform text can be sufficient when a person is the only reader. Once another system needs to act on the output, the interface needs an agreed shape. A conversational assessment of a house may be pleasant to read, but a decision buried in a session is difficult for downstream steps to retrieve reliably. Relocation Scout instead writes its assessment into a structured form.
Jones uses Karpathy’s LLM Wiki pattern as the memory layer for most of her agents. Here, the house assessment includes a decision, a score, and a reason. The wiki is a design pattern for maintained knowledge, not an installed memory product that automatically enforces an output contract; the structured record and its validation remain system design responsibilities.
The payoff is a query such as finding every house rated four or better with a commute of fifteen minutes or less. Score and commute occupy known fields instead of remaining trapped in conversation. These are search criteria, not performance measurements. The shortlist step can read those same fields without asking a person to interpret the assessment.
One step’s output is another step’s input. The contract makes that handoff dependable and forces clarity about the task itself. If the result’s shape cannot be specified, the requested work may still be too vague.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retry only the unfinished work
A reusable system must survive duplicate webhooks and interrupted runs. It needs to know whether an action already happened, whether its input changed, and which unfinished work is safe to retry. Idempotency means repeating the operation does not create unwanted additional effects.
Models add a particular trap: a retry can reword a request enough to make the same work appear to be a new task. Idempotency therefore has to be enforced by the system, rather than inferred from the model’s phrasing.
Jones illustrates this with a partially completed viewing request:
- A new listing arrives, and Relocation Scout emails the realtor about a viewing.
- The system records that the email was sent.
- The agent intends to block time on the calendar, but crashes before doing so.
- A later maintenance pass detects the completed email and missing calendar block.
- Recovery skips the email and completes only the calendar action.
This is Jones’s proposed recovery flow, not a calendar-repair capability supplied automatically by the LLM Wiki pattern. The important input to recovery is the recorded action state: an email that already went out must remain completed, even when the larger task runs again.
Recover the viewing request without sending another email
Constructed example: Object IDs and state labels are illustrative representations of Jones’s scenario. The after state depicts the described recovery outcome, not an observed execution; no listing identifier or calendar time is invented.
Ask the realtor about a viewing for the new listing and block the associated time on the calendar.
Operation: Consult the recorded action state, skip the completed email, and create the missing calendar block.
Realtor email
Sent
Sent
Email action in memory
Recorded as sent
Recorded as sent
Calendar block
Not present
Created for the viewing time
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate evidence from permission
Threat modeling starts with familiar security practices: validate inputs, grant the least privilege needed, and limit what each action can touch. Relocation Scout reads material supplied by strangers—seller descriptions, anonymous forum threads, and neighborhood reviews. That material is untrusted evidence, not a source of instructions governing the agent’s behavior.
The permission boundary follows from that distinction:
- Autonomous research: Read listings and prepare shortlists.
- Human approval required: Email sellers, book tours, or submit offers.
A listing can inform a recommendation without authorizing an external action. Keeping consequential operations behind approval reduces the blast radius if the agent mishandles an input or reaches a bad conclusion.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the system understandable from a cold start
A system can technically work and still be difficult to maintain. Jones’s concern about handing the entire design to a coding agent is that it may produce a giant prompt—or divide the work without placing responsibilities well. Understanding the architecture remains part of owning it.
Her systems include an AGENTS.md file at every level. These files explain the workflow, identify where policy lives, point to supporting skills, scripts, and sub-agents, and describe how to keep memory current. That documentation lets a human or agent become oriented without reverse-engineering a collection of prompts.
The practical test is a cold start. An agent entering with fresh context should be able to discover what to do and where to begin. Clear structure also makes it easier to ask an agent through a different harness to modify the system. Jones treats trouble making such an update as feedback: the system’s maintainability needs improvement.
Designing agents brings systems thinking, workflows, decomposition, reuse, and responsibility assignment into a new set of components. Contracts connect those components; state makes their work recoverable; safety boundaries and documentation make the whole system controllable and understandable. That is where the opening pleasure of building returns. The engineering discipline remains, with the work moved up a layer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Karpathy's blueprint for a persistent Markdown knowledge base with source ingestion, querying, maintenance, and logs.
Examples and guidance for documenting project workflows, commands, conventions, and nested instructions for coding agents.
Further reading
Jones explains how a maintained wiki supports durable agent memory through a conference research example.
The open format for packaging reusable agent instructions, scripts, and reference materials.
Full transcript
Loading transcript…