AI Engineer World's Fair 2025
Understanding is the new bottleneck
Loading slides…
Read the talk
Understanding is the new bottleneck
As agents write more code, explanations, quizzes, and custom learning environments can help developers understand enough to shape what happens next.
From a talk by Geoffrey Litt
Before you start: Familiarity with code changes and pull-request review is helpful; no knowledge of Prolog or isometric rendering is required.
How do you keep up with code you did not write?
If an agent can produce code faster than you can read it, how do you remain an effective developer? Geoffrey Litt, a design engineer at Notion, opens with the premise that people still need to understand how code works. His audience largely agrees, though he acknowledges the selection bias. The pressure behind the question is real: Litt invokes agents landing 50,000-line pull requests as an illustration of output that humans struggle to keep up with, not as a measured benchmark.
Reading every line is only one way to build that understanding. An agent can also write an explainer, generate questions that expose gaps in your knowledge, or build a small environment where you can explore the system directly. The same ability that makes code production accelerate can produce teaching materials tailored to the code you need to understand.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Correctness is only one reason to understand
The familiar reason for human review is verification. An agent proposes a change; a person checks whether it matches the specification, threatens production, or fits the architecture. These are substantial questions, but they largely lead to an acceptance or rejection decision. Understanding, in this model, is what lets the reviewer decide whether the result is correct.
Agents can increasingly ask those questions too. With an appropriate verification loop, they can check their own work and return a result that already satisfies the request. Litt welcomes that prospect: when the intention is clear, receiving a correct implementation is better than receiving an incorrect one. He expects the human role in correctness checking to decrease.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What you learn changes the next iteration
It does not follow that better automated verification removes the need for human understanding. Development is not a single loop ending in a verdict. Understanding enables participation in the next loop. When you examine a change, you come away knowing something you did not know before. That knowledge affects the next idea you have, the next question you ask, and the next direction you choose.
A rich mental model lets you recombine concepts quickly. You can imagine an alternative design or notice an opportunity without first asking another person or agent to explain every relevant part. Someone several layers removed from the implementation can still contribute, but the ideas available to them differ from those available to someone who understands the machinery. Better agents do not eliminate this connection between knowledge and creative agency.
Litt connects this to cognitive debt, a term he credits Margaret-Anne Storey with popularizing and which Simon Willison has also discussed. Like technical debt, an understanding deficit may remain tolerable for a while. A vibe-coding session can appear to be going well until you realize you no longer know what is happening well enough to participate meaningfully. The system has advanced, but your model of it has not.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn a diff into a lesson
Keeping up does not require abandoning agents or returning to an earlier pace of development. It requires treating understanding as a teaching problem. Education already offers methods for helping people acquire new conceptual structures. A raw diff supplies the changed material; it does not necessarily supply the explanation a particular reader needs. A useful design question is: what would an exceptionally good, personalized curriculum for this one change look like?
Litt’s Explain Diff skill, which he says he and coworkers use regularly, applies that question to code changes. His example is a game for drawing zen gardens. The change replaces a top-down view with an isometric perspective. The skill produces an explainer document, which can be HTML, Markdown, or a Notion page. Litt prefers the collaborative page because teammates can comment and discuss the explanation together.
The explanation begins before the change itself:
- Establish the background. Introduce the game engine, coordinate system, and relevant subsystems so the reader has the prerequisites for understanding the change. Skip or tailor material the reader already knows.
- Build intuition before showing details. Explain that the goal is to make the garden feel three-dimensional using 2D drawing tricks. Examples and a clear statement of intent give the reader something to attach the implementation details to.
- Expose the mechanism through interaction. Let the reader drag rocks in a small simulation while watching their coordinates and painting Z layers change. The movement connects the visible result to the values that determine it.
The demonstrated simulation lives inside a Notion HTML block, a feature Litt describes as having launched that morning. Interactivity needs a reason to exist: changing a rock’s position is useful because it reveals coordinates and drawing order. A widget that merely adds activity can become distracting clutter. Used selectively, interaction can convey a relationship that is difficult to grasp from static pictures alone.
Only then does the explainer reach the code. Literate code diffs introduce each file with prose and arrange the walkthrough in an order that teaches the change, rather than simply listing files. The accumulated background, intuition, and commentary make the patch easier to follow. The result can also work away from the development environment: Litt sometimes prints these documents and reads them at a café, effectively studying a small textbook about a pull request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Test understanding before requesting review
A good explanation still leaves room for false confidence. Litt once sent a pull request to a coworker believing he had read and understood it. She asked a basic question, and he could not answer. Reading had felt like understanding without producing the knowledge he needed.
That failure connects to Andy Matuschak’s Why books don’t work: readers can fail to notice what they have not understood. Matuschak and Michael Nielsen’s Quantum computing for the very curious embeds recall questions in an essay and revisits them through spaced repetition, with email reminders bringing readers back. These checks make passive reading harder to mistake for learning, though remembering an answer is not itself a guarantee of understanding or permanent retention.
Litt’s explainers end with five medium-difficulty questions. His personal rule is to pass the quiz before sending agent-written code to teammates for review. He reports that it repeatedly catches gaps he otherwise would have missed. The quiz acts as a speed regulator: progress depends on his understanding as well as the code’s correctness.
The published Explain Diff skill has HTML and Notion output versions. That makes the explanation-and-quiz workflow something a developer can adapt to their own working environment, rather than a practice tied to the particular garden demonstration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn the machine by exploring its state
An explanation teaches from outside a system. A microworld gives the learner an environment to inhabit. Litt draws on Seymour Papert’s idea of Mathland: if children learn French by living in France, what environment would let them learn mathematics through similarly direct experience? Programming the Turtle robot to draw offered such an environment. The intended outcome was a change in the child’s understanding, not merely a working robot.
Litt encountered a similar need while implementing a Prolog interpreter for his own learning. He introduces Prolog as somewhat like a database query language. Written descriptions of its mechanisms could feel complicated even when the underlying idea became straightforward once it clicked. He wanted a representation that would help those ideas become intuitive.
He had Claude build an ephemeral debugger specifically for his interpreter’s internals. Scrubbing through a timeline exposes execution step by step, with the state visible at each point. He also added comments on the timeline so he could preserve observations and remember what he had been thinking. Program, stack, timeline, and notes become parts of one environment for investigating the machine.
The debugger helped him fix individual bugs, but its value extended beyond each repair. While investigating a narrow failure, he developed a broader feel for how the interpreter behaved. Litt describes this as peripheral understanding: knowledge gained around the immediate task that would be missing if he simply delegated the fix and received the corrected code.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make a migration visible as it happens
A website migration exposed the same gap between an apparently successful result and a useful mental model. Litt asked Claude for a script to move his personal website from one framework to another. The script seemed to work, but reading it did not give him a feel for the transformation. Files had moved; he did not yet understand the process.
His next request was for something closer to a video game in which he could perform the port himself. The interface placed the old website on the left and the new website on the right. Each press of Next advanced the migration, displayed the commands being run, and made another part of the new site come to life. File trees below the previews showed files moving.
The interface preserved some of the learning benefit of doing the migration manually while removing much of the mechanical work. Instead of treating the migration as a single opaque operation, Litt could follow its sequence and connect commands to visible changes. This is a distinct use of agent-generated software: build code that helps a person understand other code. The microworld does not need to be a product intended for release; its purpose can be the understanding developed by its user.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build tools that develop the person using them
Explanations, microworlds, and shared spaces point toward an ambition larger than understanding code. Preserving people’s ability to understand how things work requires deliberate choices about what computers do for them. Litt connects that ambition to the origins of personal computing and Alan Kay’s essay A Personal Computer for Children of All Ages.
The pictured children can look, to a contemporary viewer, like children watching videos on tablets. But in the vision Litt describes, they are playing a game and changing its code to learn physics. The computer’s purpose is to develop the children’s capabilities. Litt presents AI as an opportunity to return to that educational ambition after computing’s departures from it.
Easier code generation makes ephemeral interfaces, simulations, debuggers, and playgrounds more practical to create for a particular learning need. Their value lies in what becomes understandable through using them. This possibility revives an old goal of computing: tools should expand what people can think about and do. With deliberate design, AI can put people more deeply into the work, equipped to understand it and shape its next iteration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
HTML and Notion prompts for explaining code changes through background, intuition, code walkthroughs, and quizzes.
Margaret-Anne Storey's account of how rapid development can erode a team's shared understanding.
Willison connects Storey's argument to his own experience with unreviewed generated code.
Andy Matuschak examines why reading can produce a misleading feeling of understanding.
An interactive essay by Andy Matuschak and Michael Nielsen that embeds spaced-repetition recall practice.
Further reading
Litt's written companion to the talk, with teaching examples and links.
- Notion External Agents and HTML blocksDocumentation
July 2026 release notes covering shared Claude and Cursor workflows and interactive HTML blocks.
Full transcript
Loading transcript…