AI Engineer World's Fair 2025

Understanding is the new bottleneck

Geoffrey Litt· Design Engineer, Notion19:33

Loading slides…

Read the talk

Understanding is the new bottleneck

As agents write more code, explanations, quizzes, and custom learning environments can help developers understand enough to shape what happens next.

From a talk by Geoffrey Litt

Before you start: Familiarity with code changes and pull-request review is helpful; no knowledge of Prolog or isometric rendering is required.

How do you keep up with code you did not write?

If an agent can produce code faster than you can read it, how do you remain an effective developer? Geoffrey Litt, a design engineer at Notion, opens with the premise that people still need to understand how code works. His audience largely agrees, though he acknowledges the selection bias. The pressure behind the question is real: Litt invokes agents landing 50,000-line pull requests as an illustration of output that humans struggle to keep up with, not as a measured benchmark.

Reading every line is only one way to build that understanding. An agent can also write an explainer, generate questions that expose gaps in your knowledge, or build a small environment where you can explore the system directly. The same ability that makes code production accelerate can produce teaching materials tailored to the code you need to understand.

Slide titled “How I understand agent code” with three labeled examples: Code explainers, Quizzes, and Micro-worlds.Open full source frame
Three ways to understand agent code: code explainers, quizzes, and micro-worlds.
0:220:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:22 · section reference included

Correctness is only one reason to understand

The familiar reason for human review is verification. An agent proposes a change; a person checks whether it matches the specification, threatens production, or fits the architecture. These are substantial questions, but they largely lead to an acceptance or rejection decision. Understanding, in this model, is what lets the reviewer decide whether the result is correct.

Agents can increasingly ask those questions too. With an appropriate verification loop, they can check their own work and return a result that already satisfies the request. Litt welcomes that prospect: when the intention is clear, receiving a correct implementation is better than receiving an incorrect one. He expects the human role in correctness checking to decrease.

Diagram with a faded person at a laptop and faded connecting arrows, beside a dark hard-hat figure with a circular arrow labeled “Correct?”Open full source frame
The correctness loop remains prominent as the human review loop fades.
2:282:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:28 · section reference included

What you learn changes the next iteration

It does not follow that better automated verification removes the need for human understanding. Development is not a single loop ending in a verdict. Understanding enables participation in the next loop. When you examine a change, you come away knowing something you did not know before. That knowledge affects the next idea you have, the next question you ask, and the next direction you choose.

A rich mental model lets you recombine concepts quickly. You can imagine an alternative design or notice an opportunity without first asking another person or agent to explain every relevant part. Someone several layers removed from the implementation can still contribute, but the ideas available to them differ from those available to someone who understands the machinery. Better agents do not eliminate this connection between knowledge and creative agency.

Litt connects this to cognitive debt, a term he credits Margaret-Anne Storey with popularizing and which Simon Willison has also discussed. Like technical debt, an understanding deficit may remain tolerable for a while. A vibe-coding session can appear to be going well until you realize you no longer know what is happening well enough to participate meaningfully. The system has advanced, but your model of it has not.

3:524:02
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:52 · section reference included

Turn a diff into a lesson

Keeping up does not require abandoning agents or returning to an earlier pace of development. It requires treating understanding as a teaching problem. Education already offers methods for helping people acquire new conceptual structures. A raw diff supplies the changed material; it does not necessarily supply the explanation a particular reader needs. A useful design question is: what would an exceptionally good, personalized curriculum for this one change look like?

Litt’s Explain Diff skill, which he says he and coworkers use regularly, applies that question to code changes. His example is a game for drawing zen gardens. The change replaces a top-down view with an isometric perspective. The skill produces an explainer document, which can be HTML, Markdown, or a Notion page. Litt prefers the collaborative page because teammates can comment and discuss the explanation together.

The explanation begins before the change itself:

  1. Establish the background. Introduce the game engine, coordinate system, and relevant subsystems so the reader has the prerequisites for understanding the change. Skip or tailor material the reader already knows.
  2. Build intuition before showing details. Explain that the goal is to make the garden feel three-dimensional using 2D drawing tricks. Examples and a clear statement of intent give the reader something to attach the implementation details to.
  3. Expose the mechanism through interaction. Let the reader drag rocks in a small simulation while watching their coordinates and painting Z layers change. The movement connects the visible result to the values that determine it.

The demonstrated simulation lives inside a Notion HTML block, a feature Litt describes as having launched that morning. Interactivity needs a reason to exist: changing a rock’s position is useful because it reveals coordinates and drawing order. A widget that merely adds activity can become distracting clutter. Used selectively, interaction can convey a relationship that is difficult to grasp from static pictures alone.

Only then does the explainer reach the code. Literate code diffs introduce each file with prose and arrange the walkthrough in an order that teaches the change, rather than simply listing files. The accumulated background, intuition, and commentary make the patch easier to follow. The result can also work away from the development environment: Litt sometimes prints these documents and reads them at a café, effectively studying a small textbook about a pull request.

5:566:02
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:56 · section reference included

Test understanding before requesting review

A good explanation still leaves room for false confidence. Litt once sent a pull request to a coworker believing he had read and understood it. She asked a basic question, and he could not answer. Reading had felt like understanding without producing the knowledge he needed.

That failure connects to Andy Matuschak’s Why books don’t work: readers can fail to notice what they have not understood. Matuschak and Michael Nielsen’s Quantum computing for the very curious embeds recall questions in an essay and revisits them through spaced repetition, with email reminders bringing readers back. These checks make passive reading harder to mistake for learning, though remembering an answer is not itself a guarantee of understanding or permanent retention.

Litt’s explainers end with five medium-difficulty questions. His personal rule is to pass the quiz before sending agent-written code to teammates for review. He reports that it repeatedly catches gaps he otherwise would have missed. The quiz acts as a speed regulator: progress depends on his understanding as well as the code’s correctness.

The published Explain Diff skill has HTML and Notion output versions. That makes the explanation-and-quiz workflow something a developer can adapt to their own working environment, rather than a practice tied to the particular garden demonstration.

10:0510:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:05 · section reference included

Learn the machine by exploring its state

An explanation teaches from outside a system. A microworld gives the learner an environment to inhabit. Litt draws on Seymour Papert’s idea of Mathland: if children learn French by living in France, what environment would let them learn mathematics through similarly direct experience? Programming the Turtle robot to draw offered such an environment. The intended outcome was a change in the child’s understanding, not merely a working robot.

Litt encountered a similar need while implementing a Prolog interpreter for his own learning. He introduces Prolog as somewhat like a database query language. Written descriptions of its mechanisms could feel complicated even when the underlying idea became straightforward once it clicked. He wanted a representation that would help those ideas become intuitive.

He had Claude build an ephemeral debugger specifically for his interpreter’s internals. Scrubbing through a timeline exposes execution step by step, with the state visible at each point. He also added comments on the timeline so he could preserve observations and remember what he had been thinking. Program, stack, timeline, and notes become parts of one environment for investigating the machine.

Slide titled “An interactive debugger for a Prolog interpreter,” showing green text on black with Program and Stack panels, a timeline, and populated notes.Open full source frame
A Prolog debugger displays program code, stack state, a timeline, and step notes.

The debugger helped him fix individual bugs, but its value extended beyond each repair. While investigating a narrow failure, he developed a broader feel for how the interpreter behaved. Litt describes this as peripheral understanding: knowledge gained around the immediate task that would be missing if he simply delegated the fix and received the corrected code.

11:5712:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:57 · section reference included

Make a migration visible as it happens

A website migration exposed the same gap between an apparently successful result and a useful mental model. Litt asked Claude for a script to move his personal website from one framework to another. The script seemed to work, but reading it did not give him a feel for the transformation. Files had moved; he did not yet understand the process.

His next request was for something closer to a video game in which he could perform the port himself. The interface placed the old website on the left and the new website on the right. Each press of Next advanced the migration, displayed the commands being run, and made another part of the new site come to life. File trees below the previews showed files moving.

The interface preserved some of the learning benefit of doing the migration manually while removing much of the mechanical work. Instead of treating the migration as a single opaque operation, Litt could follow its sequence and connect commands to visible changes. This is a distinct use of agent-generated software: build code that helps a person understand other code. The microworld does not need to be a product intended for release; its purpose can be the understanding developed by its user.

13:5013:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:50 · section reference included

Let the team learn in the same space

Individual understanding is not enough when a team needs to invent and build together. Effective communication depends on shared concepts: names for system components, interface elements, and the ideas behind them. A team needs opportunities to develop those concepts collectively.

One approach Litt describes is a shared chat thread containing several humans and agents. A developer asks a product manager what users want from a feature; the product manager brings an agent into the conversation to help answer. Both people can see the questions, responses, and context as they develop. Litt compares this to moving from separate one-to-one conversations into Slack channels: more of the work becomes visible to everyone participating.

A collaborative document provides a second shared surface. When Claude produces a plan, a developer can leave a comment on the relevant passage and invite a teammate into the discussion. The question and its answer remain attached to the idea being discussed, rather than isolated in a local file or private agent conversation.

Litt says his team builds substantial amounts of code within Notion because that shared context is valuable. He describes support for bringing Claude and Cursor into Notion as having launched the previous week. Those relative launch dates are his account at the presentation; Notion’s July 2026 release notes document both External Agents and the HTML blocks used earlier.

14:5815:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:58 · section reference included

Build tools that develop the person using them

Explanations, microworlds, and shared spaces point toward an ambition larger than understanding code. Preserving people’s ability to understand how things work requires deliberate choices about what computers do for them. Litt connects that ambition to the origins of personal computing and Alan Kay’s essay A Personal Computer for Children of All Ages.

The pictured children can look, to a contemporary viewer, like children watching videos on tablets. But in the vision Litt describes, they are playing a game and changing its code to learn physics. The computer’s purpose is to develop the children’s capabilities. Litt presents AI as an opportunity to return to that educational ambition after computing’s departures from it.

Easier code generation makes ephemeral interfaces, simulations, debuggers, and playgrounds more practical to create for a particular learning need. Their value lies in what becomes understandable through using them. This possibility revives an old goal of computing: tools should expand what people can think about and do. With deliberate design, AI can put people more deeply into the work, equipped to understand it and shape its next iteration.

Slide stating that humans need to understand how things work and that the right tools can help us understand better than ever before.Open full source frame
With the right tools, we can understand better than ever before.
17:0117:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:01 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…