AI Engineer World's Fair 2025
Ship Production Software in Minutes, Not Months
Loading slides…
Read the talk
Ship Production Software in Minutes, Not Months
Delegating production work takes more than a coding agent: it requires shared context, collaborative planning, reliable validation, and clear responsibility for what agents do.
From a talk by Eno Reyes
Before you start: Familiarity with Git branches, pull requests, continuous integration, and basic incident response will help you follow the workflows.
What changes when agents do the work?
What should a software development environment look like when humans no longer write every individual line of code? Eno Reyes began working with LLMs around the arrival of GPT-3.5, when agentic systems were becoming plausible. At Factory, that possibility became a product direction: move from human-driven development toward delegating work to agents across the software lifecycle.
Adding AI to an editor gives developers useful assistance, but it leaves the original interaction pattern largely intact: a human operates tools designed for humans to write code. Reyes invokes the faster-horses saying attributed to Henry Ford to ask whether improving that familiar interface is enough. Delegating most development tasks requires an environment built around delegation, rather than an accumulation of AI features.
That environment needs four capabilities working together:
- Task management: an intuitive interface for assigning work and managing agents.
- Shared context: centralized information from engineering tools and data sources.
- Reliable outputs: agents that consistently produce work worth reviewing and using.
- Parallel infrastructure: capacity for thousands of agents to work concurrently.
These are the platform requirements Reyes draws from Factory’s two years of work with large organizations, not a claim that a coding model alone supplies them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Production software still needs engineering judgment
Karpathy’s framing of English as a programming language captures the excitement around natural-language development. But being able to ask for software does not remove the engineering demands of a consequential system. Reyes illustrates the gap with a hypothetical legacy Java 7 application handling 5% of global bank transactions: it is an example of the stakes, not an identified customer or verified transaction share.
His analogy is climbing equipment. Agents make a difficult ascent more accessible, while building production software remains like scaling Mount Everest. Better equipment extends what an experienced climber can do; the climber still needs judgment about how to use it. Existing engineering expertise is what makes the new tools useful on difficult terrain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From a delegated task to a pull request
The first demonstration makes delegation concrete. Factory calls its agents Droids. Once given a task, the Droid begins by establishing where it is and what it can do; it does not immediately start editing files. The workflow proceeds in this order:
- Inspect the environment. Search the codebase, determine the Git branch, and check the machine’s available capabilities.
- Recover relevant history. Review recent code changes and retrieve memories of interactions with the user and across the organization.
- Propose and clarify. Return a plan, then ask the user to resolve uncertainties before implementation.
- Execute and validate. After the user supplies that information, write code, run pre-commit hooks and linting, and generate a pull request that Reyes describes as passing CI.
The clarification step is part of the engineering work. A useful agent should question an incomplete request and help improve it, rather than treat the first instruction as a complete specification. The endpoint shown here is a CI-passing pull request, not a production deployment.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Failure often begins with missing context
A successful demonstration raises the harder question: how do you make the outcome repeatable? Start by treating an LLM as an intelligence with only a slice of the real world available to it. What looks like a prompting problem may actually be a mismatch between what the developer knows and what the agent has been told.
Factory’s investigation of thousands of Droid-assisted sessions led Reyes to a recurring heuristic: an agent often fails because crucial context is missing, rather than because the underlying model cannot solve the problem. This is an observation from those sessions, not a universal diagnosis of agent failure. More capable models may make failures less frequent, but they cannot recover organizational facts that nobody has supplied.
The missing information may never have reached a repository or issue tracker. An LLM does not know what changed in the morning standup, what someone explained in an ad hoc meeting, or what the team drew on a whiteboard. Transcribed notes and uploaded whiteboard photos can make that information available. Native connections to engineering systems address another part of the problem, but access and use are separate requirements: the agent must be able to find the context and apply it to the task. Thinking of the system as somewhere between a coworker and a platform helps expose what it still needs to know.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Research what to build before asking for code
Using agents throughout development means involving them before implementation. In the next example, a Droid researches a new model release and plans its integration into an existing chat application. The task requires several kinds of evidence to meet in one place:
| Context | Contribution to the plan |
|---|---|
| Internet search | Up-to-date information about the model release |
| Existing codebase | How the application currently works |
| Organizational memory | The product’s goals |
| Recent design document | The technical architecture |
The difficult question is what to build within the existing product, not simply how to generate the integration code.
Planning becomes collaborative when the agent does the groundwork and research, then the developer explores possibilities with it. Asking for a finished design document in one step skips that interaction. The useful handoff is an informed starting point that people and agents can refine together.
The resulting plan can be exported to Notion, Confluence, or Jira through integrations. Reyes describes a platform experience without per-developer setup, while pointing out the friction of asking every developer to install servers, configure connections, and handle API keys. Model Context Protocol (MCP) standardizes integration interfaces; the no-setup experience is a platform claim, not a property guaranteed by the protocol. Removing that friction gets information moving, but the next improvement is organizational: standardize how the team expresses its reasoning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build the PRD from evidence and constraints
Factory’s planning for a cloud development environment feature shows what that reasoning can look like. The team started with roughly three months of feedback transcripts from enterprise and individual users. Reyes says Factory transcribes every interaction and meeting. They combined those notes with a Droid’s access to the architecture and an engineer’s ad hoc meeting notes captured with Granola.
Instead of asking the Knowledge Droid to plan the feature immediately, they asked it to investigate two things:
- Customer patterns: Which patterns in the feedback correspond to the team’s assumptions?
- Technical constraints: What limitations in the current architecture could inform a better design?
Those questions produced four or five intermediate results. The team then used the resulting documents to iterate toward a final product requirements document (PRD). The intermediate analysis matters: it gives the team evidence and constraints to examine before committing to the feature’s shape.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn the plan into coordinated work
Once the PRD exists, a Droid with Linear or Jira tools can translate it into a roadmap by creating and modifying tickets and epics. Reyes sketches eight tickets assigned across eight code Droids, with dependencies determining which work can proceed in parallel. The important operation is not merely splitting a document into tasks: it is preserving the relationships that govern when those tasks can run. The developer moves from executing each item to orchestrating the systems doing the work.
This makes ordinary process artifacts more valuable. PRDs, engineering design documents, root-cause analysis templates, quarterly engineering and product roadmaps, and meeting transcripts form a knowledge base for agents. They describe how the team thinks and works. Recording why a decision was made, alongside the decision itself, gives future developers and agents the context needed to follow the team’s practices. Documentation becomes a continuing conversation with both audiences.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Connect incident evidence before proposing a mitigation
The same context problem appears in site reliability engineering. Reyes explicitly rejects the idea that all incident response and root-cause analysis can already be automated. The demonstration has a narrower scope: a Droid takes a Sentry incident and turns it into a root-cause analysis (RCA) and mitigation plan. Producing that plan is distinct from executing a fix or resolving the incident.
Traditional incident investigation is a puzzle with pieces spread across systems: logs in one place, metrics elsewhere, historical context in documents, and some knowledge only in people’s heads. With appropriate tools, a Droid can gather relevant system logs, past incidents, runbooks in Notion or Confluence, and Slack discussions. This retrieval work supplies the evidence needed to interpret the alert.
Reyes says that access to these tools and sources can reduce incident search effort from hours to minutes. He does not provide a benchmark methodology. His next step—an agent offering an explanation and proposed fix as soon as an incident occurs—is an aspiration for immediate assistance, not a measured guarantee of zero response time. User and organization memory could then help identify recurring issues and familiar response patterns, allowing the system to propose process improvements beyond the current mitigation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make incident response a learning cycle
The RCA is an input to subsequent work. An agent can generate runbooks for newly recognized patterns, update response workflows, and capture team knowledge for automatic sharing. Connecting these steps makes incident response part of a learning cycle rather than a sequence of isolated investigations.
Reyes reports seeing unnamed teams cut incident response time in half because context is immediately available. No sample size or measurement method accompanies that report. He also describes repeated incidents prompting suggestions for underlying fixes: by the third occurrence, a Droid might flag that the recurring problem deserves a more durable solution. For a new engineer, the same organizational memory provides a way to ask how the team handles a particular situation without first finding the person who remembers it.
As operational history accumulates, patterns across incidents can inform future action. Reyes frames this as a shift from reactive toward predictive operations: each incident becomes an opportunity to improve the system’s reliability. The agent’s role is to amplify the engineer’s ability to connect evidence, recognize repetition, and improve the response process.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The work moves outward, and responsibility remains
The developer’s role expands beyond the inner loop of writing individual lines in an IDE. Reyes describes developers managing agents that work concurrently, organizing the surrounding systems, and building reusable patterns for how development happens. Working effectively with agents remains an active discipline; expecting the AI to do everything is not the same as directing it well. In that broader role, he places clear thinking and effective communication with humans and AI at the center of engineering practice.
The closing invitation was specific to the recording: attendees could scan a QR code to create an account with twenty million free tokens, and Reyes recommended a laptop because the mobile experience was not yet optimized. These were talk-time access details, not a statement of today’s offer.
Enterprise adoption adds questions that a successful code demo cannot answer. Where are the audit logs? Who owns the work? Who is responsible if an agent performs a destructive action? Reyes raises recursive deletion as a hypothetical and says Factory’s Droids do not do that; the example introduces accountability questions rather than demonstrating a safeguard.
Security, ownership, auditability, and legal indemnification all need explicit consideration. Reyes warns against unrestricted “YOLO mode” inside an enterprise and closes by inviting people to inspect the platform’s controls and send questions by email. Delegation changes who performs an action, but organizations still have to establish who authorizes it, how it can be audited, and who bears responsibility for its consequences.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The March 2025 protocol version explains how applications connect models to external context and tools.
A meeting assistant that turns conversation context and personal notes into enhanced meeting notes.
Further reading
Factory's customer account reports up to 40% lower average incident response time and separate collaboration improvements.
Updates since the talk
- Factory's context stackArticle
Explains how Factory combines repository navigation, enterprise information, and user and organization memory.
Current onboarding for delegating tasks through Factory's application, CLI, and browser interfaces.
- Factory enterprise deployment and controlsDocumentation
Current guidance on deployment, managed settings, agent safety, data flows, and telemetry.
Full transcript
Loading transcript…