AI Engineer Summit 2025
Creating Agents That Co-Create
Loading slides…
Read the talk
Creating Agents That Co-Create
From next-token prediction to shared editing environments, better AI collaboration depends on connecting model capabilities to tools people can understand, steer, and verify.
From a talk by Karina Nguyen
Before you start: Familiarity with language models, model training, and basic software development will help; the main training and interaction concepts are explained as they arise.
Why predicting the next word is not one task
Why should a model that can complete a familiar fact struggle to calculate an answer or sustain a novel’s plot? That question connects model scaling to product design: different tasks require different kinds of computation, and those differences change how people should work with AI. Karina Nguyen approaches this connection through her work on Claude during roughly two years at Anthropic, followed by research on ChatGPT at OpenAI.
Next-token prediction is a shared objective for many different tasks. Nguyen describes pre-training as learning how the world works through sequences: actions have consequences, and their order matters. Predicting what comes next can require learning those relationships. The tokens need not be ordinary words; they can represent strings or pixels as well. In this framing, pre-training becomes a form of massive multitask learning.
Some of those tasks have abundant, easy-to-learn patterns. Translation supplies recurring relationships between languages; completing “The capital of France is Paris” draws on a fact repeated throughout training material. Physics, problem-solving, logical expressions, and spatial reasoning demand more difficult relationships, with spatial reasoning still imperfect. Mathematical computation makes the contrast especially clear: generating the next token may require actually working out an answer. This uneven difficulty is Nguyen’s explanation for why scaling pre-training compute matters.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From coherent stories to useful code completion
For computational tasks, a chain of thought provides room to work through intermediate steps rather than immediately emitting the answer. Creative writing exposes a different difficulty. A model may reproduce a writing style convincingly while losing track of the world, the story, or the plot. A locally plausible continuation can damage coherence much later in the narrative.
The problem is also harder to evaluate. There is no simple measure that captures good creative writing, and reproducing familiar forms is not the whole goal: Nguyen wants models to invent new forms as well. Sustaining a coherent novel over a long span remains an open research problem. The task examples on the slide run from translation and geography through physics, spatial reasoning, mathematics, and finally writing style and plot.
Nguyen characterizes 2020–2021 as an era of extensive pre-training and points to GitHub Copilot as an early product that fit the resulting capability. Learning from billions of code tokens gives a model a basis for continuing source code. The familiar interaction is autocomplete: accept a suggestion, continue typing, and accept another, as in Copilot or Cursor.
Post-training then targets useful coding behavior. Nguyen discusses RLHF, reinforcement learning from human feedback, and RLAIF, reinforcement learning from AI feedback, as ways to shape that behavior. Her account connects these methods to the broader development of coding assistants; it does not establish the original Copilot release’s specific training recipe.
| Coding behavior | What the model must do |
|---|---|
| Function completion | Fill in a useful function body |
| Docstring understanding | Turn a description into implementation |
| Multiline completion | Continue a larger unit of code |
| Diff prediction and application | Propose and apply a change |
These tasks move beyond knowing what code looks like toward making useful changes inside a codebase. Nguyen sees substantial room for further post-training to improve reasoning across complex repositories.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
More reasoning changes the interaction
The next scaling paradigm is reinforcement learning on chain of thought, introduced here through OpenAI o1. Two distinct compute decisions matter: spending more compute during reinforcement-learning training to improve reasoning, and allowing more thinking during inference to solve a particular problem. Training uses feedback signals to teach useful reasoning behavior; test-time compute gives that behavior more room to operate.
Nguyen contrasts a conventional GPT-4o or GPT-4 response with a longer reasoning trace. Moving from translation toward difficult medical problems may require extended deliberation, access to tools, and environments in which intermediate outputs can be checked. But a visible reasoning trace introduces scientific questions of its own: how faithfully does it describe the model’s reasoning, how should that faithfulness be measured, and can the model recognize a wrong turn and backtrack?
Longer reasoning also creates an immediate interface problem. Nguyen asks how a person should interact with a model when an answer might involve a wait of fifteen seconds or thirty minutes—illustrative scenarios for the design problem. One approach her team explored was streaming summaries of model thoughts to communicate progress. These user-facing summaries are distinct from raw chains of thought. Their wording matters because they shape the user’s understanding of what the model is doing while the answer is still pending. As reasoning behavior changes, this interaction remains a design problem rather than a finished convention.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build the interaction into the learning task
An agent combines these reasoning capabilities with real-world tools—browsing, search, and computer use—over long horizons and long context. Nguyen’s proposed next stage, the co-innovator, adds creativity to that foundation. In her view, this creativity depends on human–AI collaboration, making the ways people can guide and work alongside an agent part of the capability itself.
That changes the product-research loop. Product development need not begin only after a model API is ready. Strong reasoning models can help create synthetic training data and reinforcement-learning environments, and their behavior can be distilled into smaller models that are faster to iterate on. The desired product experience can therefore help define what the next model should learn.
Consider multiplayer collaboration between people and AI. Nguyen proposes simulating different users through synthetic datasets conditioned on those users, then post-training on the resulting interactions. The sequence starts with the experience you want, translates it into a new class of tasks, and uses those tasks to shape model behavior.
Several techniques support this loop:
- Richer RL environments: Let models use search, browsing, or a collaborative tool such as Canvas while learning, so collaboration is part of the task environment.
- In-context learning: Introduce a new tool through a few examples, allowing developers to explore an interaction quickly.
- Synthetic data and distillation: Use stronger models to generate training material for models that are easier to iterate on.
- User feedback: Develop new behaviors and interactions around what people communicate while using the product.
The interface and the training task can evolve together: a new way to collaborate becomes something the model can practice.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put unfamiliar capabilities in familiar forms
A larger context window does not specify its own interface. Nguyen attributes the success of Claude’s 100K-token context window to a familiar form: file uploads. People already work with documents, so uploading one makes the capability understandable. An effectively endless chat was another possible form, but it would have presented the same underlying capacity very differently. The lesson is to find the simplest familiar action that exposes an unfamiliar capability.
ChatGPT Tasks, introduced in beta in January 2025, starts from another familiar action: scheduling something to happen later. Nguyen’s account moves from reminders and to-do lists toward work performed on a schedule. Examples include continuing a story every day and searching for topics of interest daily or every other day. She extends the idea toward language learning with multimodal, interactive visualizations; these richer compositions describe her product direction, not a list of capabilities established by the initial beta.
A feature should compose with capabilities that improve over time. Scheduling provides the timing; the model supplies the work. As the model becomes better at writing, searching, teaching, or generating interactive material, the same scheduling concept can support more valuable activities without requiring people to learn a completely new interaction.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A workspace for learning, games, and software
A flexible, multimodal workspace can also adapt how it teaches. Nguyen contrasts a person who prefers visual explanations with one who prefers audio: the model can change the presentation to suit the learner. She then offers a more spontaneous example, recounting how she used Canvas on a plane the previous day to create a game. Generative entertainment becomes something requested at the moment it is wanted.
The same possibility extends to personal tools and web applications. The slide shows a ChatGPT conversation beside a rendered Stripe Dashboard, with revenue, customer, order-value, and refund cards. Nguyen’s broader aspiration is that someone who has never written code can create a tool they need and deploy it for personal use or to start a business. The generated interface is a concrete illustration; the path from generation to deployment is the larger opportunity she describes.
Canvas can function as a pair programmer because it was trained for collaboration in both writing and coding. Search broadens what that collaborator can do: it can look up API documentation, and uploaded CSV documents can provide material for data analysis. The workspace can consequently shift from helping write prose to building software or analyzing data while preserving the interaction of asking, inspecting, and revising.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reproduction as a starting point for research
Research raises the ambition from producing a useful artifact to creating new knowledge. Nguyen shows an interaction in which she asks the model to reproduce a paper she coauthored. Reproduction supplies a concrete task around which a person and model can collaborate; reproducing an open-source GitHub repository is another example she suggests. The demonstration is a request to reproduce the work, not evidence here of a completed, validated reproduction.
From that starting point, the model can contribute its internal knowledge while the researcher helps direct the investigation. Together they can formulate hypotheses, examine research directions, and delegate particular tasks to an assistant. The shared artifact supports a continuing exchange between proposing something and checking it, rather than treating a generated document as the end of the research process.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An interface that changes with intent
Nguyen’s forecast is a layer of software creation that becomes largely invisible to its users, including people making tools directly from mobile devices. She predicts that people will click fewer links and increasingly access internet information through a model’s personalized presentation. The output need not remain text: asking to learn about the solar system could produce an interactive Three.js visualization instead.
In such a generated JavaScript scene, even a small interaction can connect the visual object to the lesson. For example, clicking an Earth mesh could select its explanation using the scene’s camera and a raycaster:
javascript
import * as THREE from 'three';
export function attachEarthLesson(renderer, camera, earth, showLesson) {
const raycaster = new THREE.Raycaster();
const pointer = new THREE.Vector2();
const canvas = renderer.domElement;
function selectEarth(event) {
const bounds = canvas.getBoundingClientRect();
pointer.set(
((event.clientX - bounds.left) / bounds.width) * 2 - 1,
-((event.clientY - bounds.top) / bounds.height) * 2 + 1
);
raycaster.setFromCamera(pointer, camera);
if (raycaster.intersectObject(earth, true).length > 0) {
showLesson('Earth', 'Explore how Earth orbits the Sun.');
}
}
canvas.addEventListener('click', selectEarth);
return () => canvas.removeEventListener('click', selectEarth);
}
This illustrates the interaction Nguyen is proposing: the learning request can become a tool the learner explores, with generated games offering another form of learning and sharing.
The larger interface metaphor is a blank canvas that changes with intent. Arrive wanting to write code, and it becomes an IDE resembling Cursor, although Nguyen leaves open how programming itself may change. Arrive wanting to write a novel, and it can generate tools for brainstorming, editing, developing character plots, and visualizing the story’s structure. The model is helping create the working environment as well as the work inside it.
The destination is creative co-direction. Nguyen envisions people collaborating with highly capable reasoning agents, including systems able to perform superhuman tasks, to create novels, films, games, and new science. In that vision, collaboration is where people and models determine what to make, revise its direction, and pursue knowledge that neither has simply been handed as a finished answer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
OpenAI's historical o1 introduction explains reinforcement learning, inference-time reasoning, example traces, and reasoning summaries.
Anthropic's 2023 announcement describes Claude's expanded context capacity and document-processing examples.
An introductory tutorial for building a browser-based 3D scene with Three.js.
Full transcript
Loading transcript…