AI Engineer World's Fair 2025
Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation
Loading slides…
Read the talk
Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation
AI simulation can turn audience research into task-directed feedback, helping designers investigate friction before deciding what to build.
From a talk by Alex Liss
What happens when users cannot trust the interface?
What happens to a product when users cannot trust what its AI tells them? Alex Liss, introducing himself as VP of Data Science and AI at Huge, starts with that problem. He cites Edelman figures matching its 2025 Trust Barometer summary: 32% of US adults say they trust AI, while 44% of adults globally say they feel comfortable with businesses using AI. These measure different attitudes, but both make trust a central constraint for AI product design.
The failures are concrete: a web search recommends eating rocks every day, and a website offers a car for one dollar. Users can recognize that something has gone wrong without understanding the model behind the answer. Putting a chatbot into an interface and presenting it as magic creates expectations the product may immediately violate.
There is a more useful capability underneath those failures. Drawing on Cassie Kozyrkov’s framing of generative AI as a UX revolution, Liss points to natural-language interaction with machine-learning models. That capability invites a design question beyond where to put a chat window: how can AI help teams understand and remove the friction users already experience?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the interface disappear
Don Norman’s The Design of Everyday Things supplies the starting principles: simplicity and efficiency. The Invisible Computer extends the ambition to software so seamless and intuitive that users practically forget they are using it. In this framing, a successful interface lets the person concentrate on their goal rather than on operating the software.
AI can contribute upstream, through needfinding: understanding the needs that should guide design. Liss presents a data scientist’s proposal for accelerating that process, rather than a portfolio of designs he has produced. The intended output is better direction for designers—insight into which problems to solve and why they matter.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn research artifacts into participants
Modern interfaces can resemble a cockpit: many screens, abundant information, and considerable complexity. Pilots learn to work in that environment through simulation and practice. For design teams, the corresponding opportunity is to practice against a simulated audience while developing an experience. The existing process already collects qualitative observations, quantitative data, and ethnographic observations to guide prototypes.
The proposed change is to make those research artifacts active participants. An AI simulation represents invisible users and gives the designer an additional feedback cycle inside the broader needfinding process. The designer can investigate an interface through that simulation, then use the resulting observations to inform further work. The simulation loop sits within research and prototyping; it does not establish that simulated behavior is equivalent to human behavior.
The overall sequence remains recognizable:
- Define the audience whose experience matters.
- Map intentions to understand what that audience wants to accomplish.
- Identify tasks that serve those intentions.
- Analyze the process of attempting those tasks.
- Refine insights from the observations.
- Develop design alternatives that address the needs uncovered.
AI simulation changes some of the participants and the workflow inside this sequence, rather than removing the need to define an audience or interpret findings.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build an audience, then give it a task
Audience simulation begins with data representing the audience. Huge’s Live platform combines demographic, psychographic, and contextual datasets. Intent mapping then turns that foundation into an Intelligent Twin: a simulation representing user behaviors, desired outcomes, needs, and motivations. The distinction matters because an audience description alone does not specify what a simulated participant should try to do.
A task brief supplies that direction. Liss compares briefing an Intelligent Twin to asking a human designer to conduct a heuristic analysis: focus the evaluation on particular tasks within a particular interface. For the sample project, he chooses a global audit of sports websites. Imagine a business considering partnerships with sports leagues across countries and cultures; it needs to understand how those leagues engage their audiences. Simulation’s proposed speed and scale make that breadth attractive, although Liss supplies no measured speedup over a human audit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A task matrix for the sports-site audit
The audit uses two contrasting personas:
- Casual fan: newer to the sport.
- Superfan: a knowledgeable, lifelong follower.
Each persona receives tasks across three websites. The tasks cover navigation, information architecture, and fan engagement, with four tasks in each category. Liss reports 72 AI-simulated actions in this sample audit: three websites × two personas × three categories × four tasks per category. This is the scope of the methodological test drive, not a count of human research participants.
The matrix can be made explicit before any simulation runs. This TypeScript expansion records the stated dimensions without inventing the task briefs; taskSlot identifies where each category’s four briefs would go.
typescript
const siteSlots = [1, 2, 3] as const;
const personas = ["casual-fan", "superfan"] as const;
const categories = [
"navigation",
"information-architecture",
"fan-engagement",
] as const;
const taskSlots = [1, 2, 3, 4] as const;
const auditPlan = siteSlots.flatMap((siteSlot) =>
personas.flatMap((persona) =>
categories.flatMap((category) =>
taskSlots.map((taskSlot) => ({
siteSlot,
persona,
category,
taskSlot,
status: "pending" as const,
})),
),
),
);
Every record is a planned task/persona/site combination. Keeping that distinction explicit prevents the size of a generated plan from being confused with completed observations or successful tasks.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From broad findings to a focused design brief
The audit can first be read at the category level, then examined more closely. Across the basketball, Olympics, and English Premier League sites, Liss reports relatively successful navigation task completion, followed by declining success deeper in content browsing, information architecture, and engagement pathways. He does not provide numerical completion rates in the spoken account.
That pattern separates reaching a website’s starting points from making useful progress inside it:
| Area | Reported pattern |
|---|---|
| Navigation | Relatively successful entry into the experience |
| Content and information architecture | Success drops deeper into browsing |
| Engagement pathways | Further friction after initial navigation |
A usable entrance does not guarantee a usable journey. Investigating the later friction gives designers a concrete place to direct their attention.
Solving those pain points is the proposed route back to trust: use AI to improve the experience people encounter. The same methodology can then move below the category-wide view to examine specific nuances of an experience. That range supports both broad briefs about recurring problems and focused briefs about particular sources of friction.
The handoff combines computer-use models, computer-vision models, and human-in-the-loop observation. The audit’s output is a design brief that helps the human team focus on distracting pain points within a category and develop responses to them. This preserves a clear division of work: simulation helps gather and organize observations; designers use those observations to decide how the experience should change.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Faster implementation makes the design question more important
Looking ahead from the first half of 2025, Liss points to Anthropic’s Model Context Protocol (MCP) and potential integrations connecting Figma prototype components to code using React, Node.js, and other frameworks. These are prospective tooling possibilities in the talk, not a demonstrated deployment. As creating an implementation becomes easier, the strategic work grows more consequential: identifying the user problem, understanding why it matters, and deciding which design deserves to be built.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the method reproducible, then test its contribution
The methodology remains experimental. Liss’s next step is a code repository that standardizes the inputs and evaluation rules needed to reproduce an audit:
- Briefing instructions: what the simulated participant is asked to do.
- Audience dimensions: which characteristics define the simulated audience.
- Audit-run counts: how many runs the evaluation includes.
- Completion and failure parameters: what determines the outcome of a task.
These are proposed standardization targets, not a released implementation described in the talk.
Once those elements are standardized, Liss proposes a test-and-control methodology to isolate where Intelligent Twins contribute to needfinding. The question is how they can complement human teams across industries, geographies, and domains. The sports audit provides an example of the process; controlled evaluation would need to establish its strengths.
The final criterion is the experience users receive. Better websites, mobile apps, and other surfaces should provide clarity and simplicity. AI simulation earns its place if it helps design teams gather useful insights faster and turn them into those improvements. Restoring trust is the intended consequence of that better experience, not an outcome already demonstrated by the sample audit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Huge's overview of audience personas and journey simulations powered by its LIVE data platform.
Updates since the talk
The June 2025 beta announcement explains how Figma supplies design context to AI coding tools.
Full transcript
Loading transcript…