AI Engineer World's Fair 2025
Designing AI to Scale Human Thought
Loading slides…
Read the talk
Designing AI to Scale Human Thought
AI can do more than complete tasks: it can expose overlooked risks, adapt to how people think, and help them develop better judgment while keeping them responsible for the work.
From a talk by Jun Yu Tan
What happens to judgment when we outsource the work?
If an AI writes your code, but you never review it carefully or understand its inner workings, how do you know what you are building—and how do you maintain the skills to judge it? That question sits behind Jun Yu Tan’s exploration of interfaces that develop human capabilities. Some of the interaction patterns are proposals rather than established product behavior, but they address a practical gap in the prevailing approach to agents.
Discrete tasks make an attractive target for automation: the unit of work is measurable, the promise of removing tedious labor is easy to explain, and completion provides a basis for comparing systems. Yet coding and design still require experience, taste, and close supervision. Much of the context needed to make a good decision lives in the minds of experienced professionals; it cannot always be packaged into a prompt. Removing the work can therefore also remove opportunities to exercise the judgment that makes the work good.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Spend compute on better human work
Augmentation uses compute to help people produce better work while retaining responsibility for it. Instead of requiring a person to accommodate an AI system’s imperfect attempt at a complex task, the interface can help them notice omissions, explore alternatives, and make a more thoughtful decision.
Consider email. An automated assistant writes the whole message and sends it on your behalf. An augmentative assistant helps brainstorm the key points, offers alternative phrasings, and checks the tone. It might notice that a sentence sounds harsh relative to your past correspondence with this recipient or your usual writing style. You decide whether to soften it, review the result, and hit send.
The same distinction applies to code. An automation workflow takes an assigned Slack message, writes the implementation, and opens a pull request. An augmentation workflow reviews a pull request you created and points out an overlooked edge case or a pattern that could fail under high load. The assistance changes what you can see before you approve the work.
The distinction is therefore partly about where responsibility remains, and partly about the relationship that develops. A thinking partner expands your field of view while you stay in control. Tan compares the difference to working with an offshore contractor versus a new teammate whose capabilities grow alongside yours.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn blind spots into reviewable evidence
The first interaction pattern, blind spot detection, asks what you have not considered. A blind spot can be temporal, such as making poor decisions when tired on Friday afternoons, or social, such as technical feedback that reads as personal criticism. Identifying it is only part of the problem. The interface also has to control noise and avoid making the recipient defensive. A useful moment to intervene is when a unit of work appears ready for review.
In the testing workflow Tan describes at Tusk, a committed code change becomes the starting point for evidence rather than a stream of speculative warnings:
- Generate unit tests targeting potential edge cases and bugs in the changed code.
- Execute those tests in the codebase’s test environment.
- Surface issues supported by the test results.
- Let the developer choose whether to commit the generated tests, including passing tests, into the pull request for future regression coverage.
Execution makes the report more concrete, but the developer still decides what belongs in the repository.
A failing test also contains assumptions about what the program ought to do. Tusk exposes its business and engineering assumptions and outlines possible fixes so reviewers can evaluate those assumptions. A thumbs-up, thumbs-down, or written explanation lets the user indicate whether an issue is worth considering and helps the system learn from that review.
Tan calls this systematic pessimism: examine every changed symbol, ask what could go wrong, and consider second-order effects through the call stack. That search becomes tedious for a human as changes grow larger and more interconnected. AI can perform the broad search while tests and explicit assumptions give the reviewer something specific to assess.
Tan reports verified bugs in 43% of pull requests and almost 1,000 new tests in two months while discussing Tusk’s work with enterprise and growth-stage customers, including DeepLearning.AI and TeamFeePay. These are reported product results, not a general benchmark for AI testing.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Allocate attention by novelty and criticality
Not every valid suggestion deserves the same presentation. The novelty–criticality framework separates how unfamiliar an issue is from how consequential it is. In code review, those two dimensions suggest four different interface behaviors.
| Novelty | Criticality | Example | Presentation |
|---|---|---|---|
| High | High | Unfamiliar race condition; customer-data exposure | Interrupt for a critical discovery |
| High | Low | New language feature; alternative approach | Offer a gentle, expandable suggestion |
| Low | High | Common bug; standard security check | Warn clearly; vary interruption by context |
| Low | Low | Formatting; minor optimization | Batch and make optional |
A familiar but serious problem still needs a warning, but it does not need a long explanation of a concept the user already understands. An unfamiliar improvement can be valuable without being urgent. Formatting preferences should not compete for attention with a data leak.
The purpose is to spend a limited attention budget well. Tan’s prioritization argument does not establish a validated numerical cap on suggestions per review. The actionable design principle is to distinguish discoveries, learning opportunities, reminders, and polish before deciding what should interrupt the user.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adapt to how the user thinks
The second pattern, cognitive partnership, moves beyond stateless answers toward a system that adapts to a person’s mental models. Knowing the desired output is not enough. The assistant may also need to understand whether someone learns through analogies or visual explanations, prefers time to reflect, or thinks through rapid iteration.
A proposed code editor could learn a developer’s refactoring patterns and suggest similar improvements in new contexts. A research assistant could learn whether its user synthesizes information chronologically or thematically, then organize its help accordingly. The useful continuity is in the way the person works, not just the facts remembered about them.
This creates a privacy constraint: personalization must make people feel understood without making them feel surveilled. A system’s ability to infer a preference does not by itself make acting on that inference comfortable or welcome.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Choose the moment to help
The third pattern, proactive guidance, makes timing central. A suggestion can be relevant and still arrive at the wrong moment. Good guidance should feel like a fortunate discovery rather than an interruption.
A calendar application might propose meeting times based on energy patterns as well as availability. A writing tool might recognize that the writer is stuck and suggest a break instead of encouraging more effort. Both proposals use context to decide when—and in what form—to intervene. The design challenge is to find a middle ground between waiting passively for every request and overwhelming the user with unsolicited help.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let trust develop through collaboration
These patterns depend on an ongoing relationship. When AI participates in how someone thinks and works, trust becomes more than confidence that a single transaction will succeed. Tan proposes three properties for that relationship.
- Progressive trust: Establish reliability through low-stakes suggestions before asking the user to rely on the system for high-impact decisions.
- Contextual trust: Keep competence specific to the domain and situation. Strong code review does not establish good design judgment; insufficient context should be visible.
- Bidirectional trust: Let both sides adapt. The AI learns preferences while the person learns its capabilities, and the system adjusts its behavior to the trust placed in it in different scenarios.
This makes trust something the interface develops and calibrates over time, rather than a blanket permission granted once.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evolve the interface with the person
If augmentation helps someone learn, the interface should change as their competence grows. It can visualize increasing expertise, introduce more complex features gradually, and adjust explanations to reflect what the user now understands. Repeating the same introductory guidance indefinitely would ignore the development the product is meant to support.
The target is real skill enhancement, not merely the feeling of being capable while the assistant is present. Learning may initially be difficult, but users should experience improvement in their core work and the satisfaction that comes with it. Increasing dependence on the tool is not sufficient evidence that augmentation is working.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure capability alongside engagement
That goal changes product evaluation. Usage, retention, and engagement still matter, but they do not establish that users are becoming more capable. A learning application can retain people without improving their performance in real-life situations. An augmentation product needs to ask how its users are growing as well as how often they return.
There are product reasons to care about that growth. First, learning a user’s preferences and intellectual needs can create a stronger relationship. Tan points to ChatGPT’s memory feature as an example of continuity across conversations; the emotional value of feeling understood is his interpretation of that experience.
Second, lasting value can come from making a team better, not only faster. Capability building gives buyers another reason to invest beyond immediate productivity. Third, users have their own stake in the outcome: time spent with an AI tool is also time spent developing—or neglecting—their abilities. Choosing tools intentionally means asking whether that investment supports learning alongside the work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Interfaces that make thought more visible
Efficiency is only part of the opportunity. Tan’s larger ambition is for AI to strengthen intuition, taste, and creativity. Blind spot detection, cognitive partnership, and proactive guidance provide possible ways to pursue that ambition, but they depend on interfaces that preserve agency, earn trust, and learn alongside their users.
The next-decade vision is an aspiration: tools that help people think in ways they could not before. An interface can act as a mirror for cognitive patterns, making habitual omissions visible and suggesting perspectives a person repeatedly misses. The value lies in what the person comes to notice and understand, as well as what the system produces.
How to build such systems remains an early design problem. The durable criterion is whether an interface enhances human capability as understanding of the technology evolves—whether using it helps people become more thoughtful, more creative, and more able to judge their own work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
OpenAI's memory announcement explains personalization across conversations, user controls and dated rollout updates.
Further reading
Jun Yu Tan's essay on cognitive partnership, progressive trust and evaluating growth in human capabilities.
Tusk's customer case study describes backend test generation and reports edge cases identified in 42.9% of testable pull requests during the first two months.
Updates since the talk
- Tusk Is SunsettingArticle
Official notice of Tusk's June 10, 2026 product sunset and transition guidance for existing customers.
Full transcript
Loading transcript…