AI Engineer Europe 2026

How to talk to statues — Joe Reeve, ElevenLabs

Joe Reeve33:28

Loading slides…

Read the talk

How to talk to statues

A photograph becomes a speaking museum object, then opens a harder design problem: how to give voice agents useful knowledge, conversational cues and interfaces people can steer.

From a talk by Joe Reeve

Before you start: Basic familiarity with APIs and chat-message history helps with the implementation discussion; no audio-model background is required.

What would a statue sound like?

What would it take to photograph a statue and have a conversation with it? The necessary pieces already exist as audio services. Joe Reeve, who works in ElevenLabs’ growth organization, introduces text-to-speech, transcription, sound effects, voice creation and editing, and a managed agents platform. He singles out voice design as an underused capability: a developer can describe a voice instead of selecting an existing one. He also presents music generation as commercially licensed; the Eleven Music API announcement describes broad commercial use subject to plan restrictions, rather than unrestricted clearance for every use.

The statue app makes that collection of capabilities tangible. Its introductory slide pairs an interface identifying Artemis the Huntress with a photograph of the British Museum entrance. Reeve then plays a museum visit in which objects introduce themselves: Pharaoh Amenhotep, Demeter, Hoa Hakananai'a, Hans Sloane, the Guardian Lion, the Young Rider and the War Horse. A collection that visitors normally look at becomes a collection they can address.

Slide titled “Who saw this?” with a social post, an app screen identifying Artemis the Huntress, and a photograph of the British Museum entrance.Open full source frame
The statue app introduction pairs an Artemis interface with a British Museum photo.
0:360:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:36 · section reference included

From a photograph to a conversation

The app connects the services in a short sequence:

  1. Photograph the object. Use the image to identify the statue.
  2. Research its identity. OpenAI supplies historical knowledge and descriptions of what the statue’s voice might sound like if it were alive.
  3. Generate a voice. The Voice Design API turns a textual description into a matching synthetic voice.
  4. Create an agent and start a call. Combine the persona’s knowledge and voice in an ElevenLabs agent, then open the conversation.

Reeve reports roughly 30 seconds from taking the photograph to talking with the agent. That is his end-to-end example, without a supplied timing distribution or model configuration.

“How it works” slide lists taking a photo, identifying it with a multimodal LLM, researching the statue, generating voices with the Voice Design API, creating an ElevenAgent, and starting a call.Open full source frame
From taking a photo to starting a call: the statue app workflow.

The QR code leads to Reeve’s companion build tutorial. His spoken description calls the research step deep research and the resulting interaction a phone call. The subsequently updated tutorial uses GPT-5.2 with web_search_preview and a browser WebRTC conversation; it does not establish the exact historical research model or a telephone-network connection for the demonstration. The enduring architecture is image identification, research, voice generation and agent creation.

2:222:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:22 · section reference included

A fast prototype meets institutional demand

Reeve reports building the prototype in Cursor in two hours on a Sunday and publishing its build prompt through the ElevenLabs blog. The first video attracted interest from museums and travel businesses, including Tripadvisor competitors. He says three museums or museum-group representatives contacted him; one CEO told him a team of ten had spent a year on a similar idea. These were accounts of interested organizations, not evidence that the prototype had already met their requirements.

The next post emphasized how quickly he had built it. Reeve reports that impressions grew from 50,000 on the first day to 1.5 million on the second after the vibe-coding repost. The response expanded into the arts: Bonhams and Christie’s wanted people to converse with items offered for sale. The experience also helped inspire ElevenHacks. What began as an API experiment had become a question about cultural experiences, distribution and what people could build for themselves.

An audience member asks the obvious engineering question: how does this become a production service for millions of users? Reeve’s answer separates the underlying services from the application around them.

LayerPrototype’s approachRemaining responsibility
Voice generation and agentsManaged APIsDepend on provider capacity and operation
User accountsPotential third-party serviceImplement and check authentication
Product integrationGlue between servicesFinish the museum experience

Reeve expects the voice and agent APIs to have ample capacity for this app. He does not demonstrate a million-user deployment. His point is that useful integration and a compelling explanation can create substantial value without rebuilding the underlying infrastructure.

Production work still remains. He is discussing a possible ElevenLabs-supported offering with museums, and suggests Supabase logins and magic links as conventional application pieces Cursor could help implement. Generated code still needs checking. Managed APIs remove major infrastructure tasks; they do not finish the product or decide what a museum should put into it.

3:223:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:22 · section reference included

The museum must author the experience

Asked about evaluations, Reeve turns to curatorial authority. Photographing an object and assembling web research is useful for the prototype, but he does not see it as the long-term museum workflow. Curators need to decide the narrative and design the content. Museums already regard their collection databases as core intellectual property, and some expose that information through APIs, including the V&A Collections API. His answer identifies the source-of-truth problem; it does not supply an evaluation suite or measured results.

That raises a second question: where does a curator do this work? No dedicated curator interface has been built. The available route is the ElevenLabs dashboard, where a curator could edit the system prompt and knowledge-base files. Reeve favors text editing for information management, even when the visitor-facing experience is spoken. Choosing the voice is a separate design task.

A voice description also encodes an interpretation of the object. Reeve describes conversations with Jago Cooper, formerly head of the Americas at the British Museum and now director of the Sainsbury Centre—the building used as the Avengers headquarters in the films. Cooper, he says, is investigating what inanimate objects ought to sound like. Consider the illustrative journey Reeve offers: stone originates in China, is carved in Vietnam, and spends two centuries in a British museum. Should its voice reflect the material’s origin, the place of manufacture, its later surroundings, or the tourists who pass it?

There is no settled answer, and the question extends beyond museums. A lift already has a physical character, yet its recorded voice can feel disconnected from it. If passengers begin requesting floors through conversation, choosing that voice becomes part of designing the object itself. Voice generation makes the choice possible; it does not make the cultural or aesthetic decision automatically.

7:057:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:05 · section reference included

Talk to the right agent, with a screen alongside it

Voice interfaces often force a choice between speaking and using another interface. Reeve wants them to work together. His coding example is a Lovable-like app in which the user talks to a product-manager agent, which then delegates implementation to a coding agent. The conversational partner and the worker need not be the same component: one helps establish what should happen, while another carries out the task.

A companion interface could display information extracted from the conversation and let the user edit it while speaking. That makes the conversation inspectable and correctable without requiring every correction to become another spoken exchange. There is also a social obstacle: people hesitate to interrupt an agent. Reeve finds that interrupting more aggressively improves his experience, but has no settled way to give other users permission to do it.

The next question concerns skills and prompt guidance. In the platform Reeve describes, ElevenAgents has no native skills concept, but it can load knowledge files and make MCP calls. Those are possible routes for supplying knowledge or exposing capabilities. More generally, voice can be attached to an existing coding agent rather than requiring the voice platform to reproduce its entire working environment.

An audience member makes the desired interface concrete: speaking to Claude Code in a human voice. Reeve reports that people have connected ElevenLabs to OpenClaw and some Claude Code setups, including phone numbers they can call and agents that call back. He also describes adding skill-loading capability through the agent itself as a possibility. These are examples of integration patterns, not a live implementation shown in the session.

9:5010:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:50 · section reference included

Make the technology part of the experience

Museum engagement remains an open design problem even for museum leaders. Reeve describes discussions with Science Museum leadership and its willingness to experiment with interactive tablets for children. To him, a screen beside an exhibit can still feel like technology attached to an existing experience. He reports commissioning a statue with embedded technology, a speaker, a phone and a microphone so visitors can speak directly to it. At the venue, a K6 red British telephone booth offers another physical interface: pick up the handset and talk with a Sir Michael Caine agent.

An audience member extends the idea from consuming an exhibit to creating an experience: a child could arrive at a science museum and build something in response to what they encounter. Reeve thinks consumer creation has not yet reached that point. Even tools aimed at nontechnical users can feel oriented toward B2B SaaS applications, with Supabase and familiar design components determining the shape of the result.

Evening vibe-coding events offer a different picture. Reeve meets people who have never considered that someone makes the apps on their phones, much less learned what an accordion or hamburger menu is. They describe what they want, and an LLM attempts combinations an engineer might immediately translate into familiar components. Their lack of software vocabulary can allow unusual ideas to survive longer. He wonders what an Instagram-filters or TikTok moment for social software creation would look like.

He mentions a mobile game-creation app he calls Spielwork, with TikTok-style swiping, and an unnamed London game-building tool. Games may themselves be too complex an entry point, he suggests, and many offerings still resemble Lovable on a phone. The audience broadens the goal to content creation, building and interacting with digital systems—including spawning game objects by voice. A mainstream creator should be able to express an intention without first adopting concepts such as databases.

13:1913:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:19 · section reference included

Creation needs distribution primitives

Reeve’s historical template is Facebook Instant Games in Messenger: small games such as quizzes or Fruit Ninja, played competitively within group chats. He describes that experience as deprecated; this does not establish that the entire Instant Games API has ceased to exist. The useful pattern is the compact set of social and commercial primitives exposed through JavaScript’s async/await style.

PrimitiveWhat it supplies
User information and friendsIdentity and social connections
LeaderboardsCompetition and visible standing
Key-value storageBasic persistence
Rewarded and interstitial adsMonetization

Together, these let a small game participate in an existing social network instead of requiring its developer to build accounts, sharing, competition and advertising from scratch.

Reeve says he bought a Fruit Ninja clone for £15, integrated Facebook Instant Games, and woke the next day to 15 million users in Vietnam. He reports little revenue, and attributes the geography to Facebook testing in lower-value advertising regions. The anecdote illustrates why he sees social connections and immediate sharing as promising ingredients for consumer vibe coding: producing a creation is only part of the experience; putting it in front of friends gives it somewhere to go.

17:4817:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:48 · section reference included

Freeform speech in, dense information out

An audience member identifies an asymmetry: speaking is a convenient way to get thoughts out, but listening can be an inefficient way to receive the result. They want to express raw intent and see it transformed into diagrams, text or application state. Email is one desired setting: speak freely, then inspect an information-rich result rather than wait through a spoken explanation.

Reeve wants the same arrangement: one voice input, parallel outputs. A little speech might accompany a generated interface, a diagram or changes in the current application. Another audience contribution adds a different benefit of spoken output: companionship. A voice can make tinkering feel less lonely and sustain motivation even when text or diagrams would convey more information. The choice of output therefore depends on both the information being delivered and the experience surrounding it.

19:2119:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

19:21 · section reference included

Navigate spoken answers by concept

The discussion briefly reaches for an evolutionary explanation of visual processing, but the practical usability observation is more immediate: concise text and concise speech can feel different. One participant is happy to request a short written answer, yet finds the equivalent spoken answer abrupt or rude. Simply reducing word count does not solve the listening problem.

Reeve proposes skim listening. Forward and backward controls could initially jump through pieces of an answer, much as podcast speed controls or an old iPod’s scrubbing interface let listeners move through audio. The discussion then refines the unit of navigation: a listener wants to move between concepts or topics, not necessarily sentence boundaries.

That distinction matters because a spoken navigation request can accidentally become a generation request. If an agent is reading a long answer and the user asks it to move to the next point, it may treat the interruption as a new prompt and generate another long explanation of that point. The intended operation was to advance through an existing answer, not expand it.

The room suggests several ways to expose the answer’s structure:

  • Topic navigation: move directly to the next concept.
  • Summary and expansion: show a short paragraph summary, then hold a control to hear more.
  • Separate visual and spoken content: display higher-level sections while speech develops each one.

Reeve describes Claude’s voice interface as exploring the last pattern with tappable sections. These controls move beyond the conventions of ordinary human conversation, but they give a listener direct access to information that otherwise unfolds only over time.

21:3921:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

21:39 · section reference included

Make turn-taking visible

Human conversations carry signals that audio-only agents lose. A listener can often see that another person is about to interrupt, and a speaker can accelerate, change direction or notice disengagement. With an agent, the listener may not even know whether the answer will take ten seconds or a minute. Advance information about its length could help them decide whether to continue or cancel. The interaction resembles a phone call in its missing visual cues.

Even incoming speech has different meanings. Acknowledgments such as “uh-huh” or “yeah, yeah” can encourage a speaker to continue. A request to go back is steering. Neither is necessarily a demand to stop the current turn and begin a completely new exchange. A voice interface needs to preserve those distinctions.

An audience developer describes a game in which a player can request a getaway vehicle or helicopter while participating in a shooter-style encounter, then use the generated object in the world. Automatic voice interruption proved unreliable for their work, so they adopted a Wispr Flow-style push-to-talk boundary: hold to speak, release to finish. A physical input supplies a clear signal that the audio stream alone was failing to provide.

For the opposite direction, the developer proposes a small visual indicator showing that the agent wants to ask a question. The user could finish speaking or invite it to contribute, without being interrupted immediately. Reeve extends the idea: show the topic the agent wants to raise. That would communicate more than a human’s preparatory gesture—the user could decide whether that particular contribution is worth taking now. These are proposed interface behaviors, not features demonstrated in the session.

23:4823:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

23:48 · section reference included

Conversation history should reflect what was heard

How would an agent decide that it has something to add? An audience member asks whether this requires a tool call, RTC audio handling, a separate channel or changes to the system prompt. Reeve’s proposed approach is an asynchronous transcript monitor: repeatedly inspect the evolving conversation and ask whether there is a useful contribution to make. He explicitly describes this as something still to build, rather than an existing transport or tool-calling design.

The follow-up question is how to steer an agent that already has a plan. Reeve separates the agent into next-message logic and a malleable conversation transcript. In the behavior he describes, a platform generates an answer’s text and then plays its audio. If the user interrupts partway through playback, the platform may append the interruption after the entire generated answer—even though the user heard only its beginning.

Generated text and heard speech are different states. Reeve suggests using playback timestamps to locate the interruption and remove the unheard suffix from conversation history. For a concrete example, suppose an assistant generated First, consider the material. Next, consider the carving. Finally, consider the museum. The listener hears only the first sentence and says Go back to the material. The proposed history repair keeps that first sentence, removes the unheard remainder and appends the interruption.

Repair history at the interruption point

Constructed example: The message IDs, system instruction, assistant text, user interruption and first-sentence playback boundary are teaching values. The transcript-repair mechanism is Reeve's unbuilt proposal.

Interruption after the first sentence — unchanged
Go back to the material.

Operation: Proposed repair: retain the heard prefix of assistant-1, remove its unheard suffix and append interruption-1.

System message

Before: Generated history before repair
Explain the object’s history.
After: Proposed history for the next response · Unchanged
Explain the object’s history.

Assistant message

Before: Generated history before repair
First, consider the material. Next, consider the carving. Finally, consider the museum.
After: Proposed history for the next response · Changed
First, consider the material.

User interruption

Before: Generated history before repair
Not present
After: Proposed history for the next response · Added
Go back to the material.
The next response should not assume the listener heard the generated suffix.

The following JavaScript expresses that history operation once playback alignment has supplied heardTextEnd, the exclusive character boundary of the text heard. It requires the interrupted assistant message to be the latest history entry, so unrelated messages cannot be silently removed.

javascript

function recordInterruption(history, messageId, heardTextEnd, userText) {
  const last = history.at(-1);
  if (!last || last.id !== messageId || last.role !== "assistant") {
    throw new Error("Expected the interrupted assistant message last");
  }
  if (!Number.isInteger(heardTextEnd) ||
      heardTextEnd < 0 || heardTextEnd > last.content.length) {
    throw new RangeError("Invalid heard-text boundary");
  }

  const heardText = last.content.slice(0, heardTextEnd).trimEnd();
  return [
    ...history.slice(0, -1),
    ...(heardText ? [{ ...last, content: heardText }] : []),
    { id: "interruption-1", role: "user", content: userText }
  ];
}

const history = [
  { id: "system-1", role: "system", content: "Explain the object’s history." },
  {
    id: "assistant-1",
    role: "assistant",
    content: "First, consider the material. Next, consider the carving. Finally, consider the museum."
  }
];

const repairedHistory = recordInterruption(
  history,
  "assistant-1",
  "First, consider the material.".length,
  "Go back to the material."
);

The timestamp-to-text alignment is the essential input; a fraction of elapsed audio time is not necessarily the same fraction of the text. This example implements the proposed transcript edit, leaving playback cancellation and subsequent response generation to the surrounding application.

An audience member suggests treating such user messages as a distinct form of steering. Reeve’s response is to invite a prototype at the ElevenLabs booth. The practical experiment is small but consequential: make the next response depend on the conversation that actually reached the listener, while giving navigation and acknowledgment their own meanings.

26:4926:59
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

26:49 · section reference included

Make the demo worth watching

The final question returns to distribution: when building becomes easier, how do people discover what you made? Reeve learned video production through political campaigning before joining ElevenLabs, and that experience fed into ElevenHacks. He finds virality unpredictable. Practice and iteration are more dependable than confidence that a particular video will succeed.

He applies an 80/20 heuristic to editing: reaching the statue video’s level of polish was relatively easy, while going further would introduce substantially more work. Reeve reports spending about 20–25 minutes editing the statue video on his phone. He finds desktop CapCut much harder to use and describes its subscription as three times the phone price in his experience. The practical choice is to keep the workflow simple enough to repeat.

Captions help, and the interesting part needs to arrive early. Reeve reports median viewing times of 6–12 seconds across his videos. His advice is to inspect the platform’s analytics after publishing several examples, then put the hook before the point where viewers tend to leave. A compelling demonstration buried late in the edit will never reach much of its potential audience.

Music can be developed in either direction:

  • Narrative first: edit the speech and story, generate different musical genres, then adjust musical sections to fit the spoken sections.
  • Music first: choose the feeling, generate a track, and build the speech and edit around it.

For the statue video, Reeve chose the music first. Its imperial feeling suited the British Museum exterior and served as an attention grabber. Even quiet background music, he finds, strongly changes the feeling of a scene.

The equipment stayed simple: a mobile phone, CapCut and a borrowed DJI Bluetooth lapel microphone that improved the recorded audio. Reeve says the microphone cost £200. He deliberately planned the shot outside the British Museum, but the finished video came from a second recording trip. The first visit produced footage he found boring; he went back, took a collection of silly photos, and those became the video people saw. The iteration was in the demonstration itself as much as in the software.

29:1129:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

29:11 · section reference included

Resources

From the talk

  • Eleven Music API launchArticle

    The August 2025 announcement explains music generation capabilities and plan-dependent commercial usage.

Transcript download
Full transcript

Loading transcript…