AI Engineer World's Fair 2025
Designing AI-Intensive Applications
Loading slides…
Read the talk
Designing AI-Intensive Applications
From conference feedback loops to AI News’s daily pipeline, reusable application patterns offer a more useful design vocabulary than arguing over what counts as an agent.
From a talk by Shawn Wang (swyx)
Before you start: Basic familiarity with language models and application workflows is helpful; the TypeScript example uses asynchronous functions and promises.
A conference that tracks its field
How do you organize a conference for a field whose interests keep changing—and whose attendees keep registering late? Opening the June 2025 AI Engineer conference, Shawn Wang reports 3,000 attendees and jokes about measuring last-minute registration through a Gini coefficient of organizer stress. The comparison with the previous year is a joke about timing, not a supplied numerical coefficient. It introduces a practical problem: responding to demand while building an event that reflects where AI engineering is going.
Wang says the conference doubled every previous year’s track. That expansion buys breadth at the cost of more simultaneous choices and attendee decision fatigue. The ambition is to cover the field, even when nobody can attend everything.
Programming also has a feedback loop. The organizers aim to respond faster than research conferences such as NeurIPS while remaining more technical than general-interest events such as TED. They survey attendees about topics—including computer-using agents and AI with crypto—and use those responses to shape the program. At the opening, the survey is still running: wider participation should produce a more representative sample, with results promised the following day and responses informing the next year’s event.
The conference itself is an application surface. Wang describes it as the first conference with an MCP interface and the first to accept an MCP talk through MCP. He credits Sam Julien from Writer for the official chatbot, Quinn and John from Daily for the official voice bot, and Elizabeth Trykin from Vapi for helping prototype the voice experience. These are attempts to improve the attendee experience by putting the field’s tools to work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A growing field does not require elaborate scaffolds
The changing conference structure follows a changing job. In 2023, Wang organized his opening talk around three types of AI engineer. In 2024, the emphasis shifted to multidisciplinary work, reflected in the World’s Fair’s multiple tracks. Early 2025 in New York brought a focus on agent engineering. By this June gathering, the once-dismissive label “GPT wrapper” sits awkwardly beside the commercial success of applications built around models. Wang’s joke that everyone is now rich captures that reversal in status, rather than establishing a financial result.
Increasing capability does not imply increasing scaffold complexity. Wang points to Erik Schluntz’s Latent Space discussion of Anthropic’s SWE-bench success with a simple scaffold. The point is a strong benchmark result from a relatively simple system, not that the benchmark was completely solved. Greg Brockman’s discussion of Deep Research and Amp Code supply related examples. The field is still young enough that useful discoveries need not depend on elaborate infrastructure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What would a standard model of AI engineering explain?
Wang compares this formative moment to the 1927 Solvay Conference, where Einstein, Marie Curie and other major figures in physics gathered. The analogy is about bringing a developing field’s practitioners together while its basic ideas are taking shape. He then points to the physics standard model: in his telling, a concentrated period of development from the 1940s through the 1970s produced a framework with decades of durability. The corresponding engineering opportunity is to identify ideas that will remain useful after today’s products and terminology change.
Software already has this kind of shared vocabulary. ETL describes moving and transforming data; MVC separates interface concerns; CRUD names common record operations; MapReduce organizes distributed computation. These patterns help engineers recognize a problem and discuss a solution without starting from scratch. AI engineering has RAG, but one pattern cannot describe the whole application. Wang treats recurring claims that long context or fine-tuning will kill RAG skeptically. The useful question is which additional models can guide design.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Architecture and lifecycle describe different concerns
Several candidates address different parts of the problem. Wang names DSPy and points attendees toward Omar’s talk, then starts with LLM OS, which he attributes to Andrej Karpathy in 2023. His 2025 update incorporates multimodality, an emerging standard tool set and MCP, which he characterizes as the default protocol for connecting to the outside world. This is an architectural lens: it places the model alongside the capabilities and connections an application needs.
The LLM software development lifecycle offers another lens. Wang mentions two versions, including one organized around the intersecting concerns of purchased tooling, and directs readers to the Latent Space blog. The consequential distinction comes from a conversation with Ankur of Braintrust: getting started is becoming increasingly commoditized, while making an application dependable for customers remains substantial engineering work.
| Lifecycle concern | Practical distinction |
|---|---|
| LLM access, monitoring and RAG | Free tiers lower the cost of getting started |
| Evaluations, security and orchestration | Hard engineering becomes central when serving customers |
“Free” here means access through free tiers, with payment beyond those limits—not zero operating cost. Wang associates the latter concerns with moving from demos into production and explains that the conference added tracks to support that transition.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Agent definitions leave some useful applications unexplained
Anthropic’s Building effective agents supplies another candidate model. Co-author Barry had presented it at the previous conference, and Wang describes it as received guidance for building agents. Its architectural distinction separates predefined workflows from processes in which the model directs execution and tool use. OpenAI uses a different definition, illustrating that the vocabulary is still evolving.
Wang also mentions a recent Agents SDK improvement from Dominik, described as building on OpenAI’s swarm concept. The linked OpenAI Agents SDK repository documents the current project; its present API should not be read as a snapshot of that June 2025 announcement. Wang’s own earlier Agent Engineering framing takes a descriptive approach, organizing the words people use around intent, control flow, memory, planning and tool use. But that vocabulary has a practical limit: he does not use all of it to build AI News.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Human input versus valuable AI output
Wang reports that AI News had more than 70,000 readers at the time of the talk. It began as a tool he wanted for himself. At the previous conference, Soumith, introduced as PyTorch’s lead, told him he enjoyed reading it but did not consider it an agent. Wang initially resisted, then accepted the distinction: it is a workflow. Its usefulness survives that change in label.
Compare the human input required with the valuable AI output produced. This shifts attention from whether an application deserves an agent label to what the user gets from using it. Copilot’s interaction is tightly coupled to typing: a debounced trigger produces an autocomplete after a few characters. ChatGPT moves to a query-and-response interaction. Reasoning models suggest more work from a single request; Wang illustrates this with a one-to-ten input/output ratio, a conceptual comparison without a defined measurement unit or benchmark.
Deep Research and NotebookLM extend the idea toward larger outputs from an initial request. Wang points to Raiza Martin’s product-management talk for the NotebookLM story. Ambient agents stretch the comparison further: can a system produce useful output without a new human request? His zero-to-one framing describes that interaction pattern, rather than the absence of setup, permissions or operating costs. The completed slide’s ratios are a mental model for these interactions, not measured productivity multipliers.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The repeated workflow behind AI News
AI News makes the alternative concrete. Wang describes it as scripts in a trench coat and recognizes that he has written essentially the same system three times: once for Discord, once for Reddit and once for Twitter. Each source repeats the same sequence:
- Scrape the source material.
- Plan the processing work.
- Recursively summarize the material.
- Format the result.
- Evaluate the output.
Recursive summarization is the crucial many-to-one operation: intermediate summaries can themselves become inputs to further summarization, bringing a large collection toward a consolidated result.
The workflow runs daily, with incremental improvements alongside Wang’s work organizing the conference. Repetition across sources suggests a reusable application pattern. An AI-intensive application can coordinate thousands of AI calls toward a single purpose; that is the scale of the generalized design he proposes, not a measured daily call count for AI News.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
SPADE: from many inputs to one delivered result
Generalizing that repeated workflow produces SPADE. Sync brings material into the system; planning organizes parallel processing; analysis reduces many intermediate results into one; delivery puts the result in the user’s hands; evaluation assesses it. The pattern separates collecting information, doing work over it and presenting the result, without requiring the model to choose the entire control flow.
A TypeScript orchestration function can express those stage boundaries directly. Here, the stage implementations are supplied by the application, while the function preserves the sequence and parallel processing:
typescript
type SpadeStages<Input, Job, PartialResult, Result, Output, Evaluation> = {
sync(): Promise<Input[]>;
plan(inputs: Input[]): Promise<Job[]>;
process(job: Job): Promise<PartialResult>;
analyze(parts: PartialResult[]): Promise<Result>;
deliver(result: Result): Promise<Output>;
evaluate(output: Output): Promise<Evaluation>;
};
async function runSpade<Input, Job, PartialResult, Result, Output, Evaluation>(
stages: SpadeStages<Input, Job, PartialResult, Result, Output, Evaluation>
): Promise<{ output: Output; evaluation: Evaluation }> {
const inputs = await stages.sync();
const jobs = await stages.plan(inputs);
const parts = await Promise.all(
jobs.map((job) => stages.process(job))
);
const result = await stages.analyze(parts);
const output = await stages.deliver(result);
const evaluation = await stages.evaluate(output);
return { output, evaluation };
}
For AI News, the inputs come from the scraped sources, the processing includes summarization, and delivery formats the consolidated result. Promise.all makes the parallel stage visible; at the thousands-of-calls scale, an implementation also needs to bound concurrency to its service limits.
The stages admit richer representations than a text-in, text-out pipeline. Material can become a knowledge graph, and model results can become structured outputs. Delivery can produce generated code rather than prose: ChatGPT Canvas and Claude Artifacts illustrate that possibility. The completed slide connects the stages to planning and read tool calls, summarization and write tool calls, code generation and generative UI, and evaluation with an LLM judge. These are possible techniques within the pattern, not requirements that every application adopt them all.
SPADE remains a developing mental model, with the detailed implementation deferred. Its value is in making a recurring shape recognizable: gather many inputs, coordinate work over them, consolidate the results and deliver something useful.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adding intelligence people want to use
The test for any of these models is whether it helps an application take its next useful step. Adding intelligence should help the user without becoming annoying. LLM OS, lifecycle models, agent frameworks and SPADE are hypotheses about how to do that, not settled standards.
Wang’s closing invitation is to use the conference’s conversations to discover patterns that other engineers can reuse. A shared model earns its place when it helps people improve their applications—and, echoing Laurie’s opening point, build products people want to use.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Podcast and transcript explaining Anthropic's coding-agent scaffold, tool design and SWE-bench results.
Anthropic's guide to workflows, autonomous agents and simple composable application patterns. The live page includes later tooling updates.
Browse the published output of the AI news aggregation project used as the talk's central application example.
Documentation for programming and optimizing language-model applications with DSPy.
Further reading
Technical account of a 49% SWE-bench Verified result, with the prompt, tool specifications and evaluation limitations.
- Agent EngineeringArticle
Companion to swyx's earlier keynote, organizing agent definitions around intent, control flow, tools, planning, memory and authority.
Updates since the talk
Current SDK source and examples for agents, tools, handoffs, guardrails and tracing.
Full transcript
Loading transcript…