AI Engineer World's Fair 2026

The Next Medium: Why Real-Time Interactive Video Changes Everything — Ahmed Ahres, Reactor

Loading slides…

Read the talk

The Next Medium: Why Real-Time Interactive Video Changes Everything

Real-time video generation replaces the finished file with a session people can steer, creating new applications and new demands on streaming, memory, latency, and evaluation.

From a talk by Ahmed Ahres

What if you could change the video while watching it?

A prompt produces a video file. You watch it, decide what you wanted differently, and try again. That is the workflow Ahmed Ahres contrasts with real-time interactive video. He uses world models to mean this live, steerable medium, rather than other uses of the term associated with techniques such as Gaussian splatting.

In the prompt-to-file workflow he describes, Veo 3 and Seedance 2 can generate impressive footage, but the returned recording cannot respond to you while it plays. The question is what happens when pixels become programmable like software: generated continuously, with inputs that can change what appears next.

Slide contrasts passive video with programmable video and pixels generated in real time.Open full source frame
What happens when video becomes programmable, like software?

Ahres introduces himself as Reactor’s head of go-to-market, with a background in computer vision and machine learning, experience as a founder, and a hobby of shipping iOS and Android games. He describes Reactor as a Series A company building the infrastructure and developer platform that make these models accessible to application builders. The underlying problem is straightforward: generating a recording does not make it interactive. Real time changes the medium, not just the speed of production.

0:310:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:31 · section reference included

Immediate feedback creates different applications

A printed map gives you a representation someone prepared earlier. GPS adds a continuously updated position: you can see where you are and track movement. In Ahres’s analogy, the consequence is larger than locating yourself faster. Live location becomes an input to an application such as Uber.

Photography supplies a second feedback loop. With film, you shoot first and see the developed result later. A digital camera or iPhone shows what you are recording as you record it, letting you adjust the shot immediately. Ahres connects that ability to the quality of everyday content creation and, in turn, to Instagram and TikTok, while acknowledging that other factors contributed. The useful mechanism is the shortening of the loop between action, observation, and adjustment.

Three connected cards labeled Film you sent away, The live viewfinder, and TikTok and Instagram beneath a heading about seeing in real time.Open full source frame
From film to the live viewfinder to TikTok and Instagram.
2:593:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:59 · section reference included

Steering a continuing video

Programmable video can be addressed, conditioned, and changed. The first model category resembles Veo or Sora, but with a continuing stream instead of a fixed-duration result. Ahres describes three properties together: generation continues, inputs can alter it, and the viewer sees the output as it is generated. His description of an infinite stream concerns continuing generation; it does not establish unlimited reliable memory.

The demonstration starts with an image of a dog. While the video continues, Ahres introduces the prompt “A cat shows up” and describes a cat appearing in the generated scene. The intervention happens during generation, rather than requiring a replacement for an already returned file. He then proposes further directions—a running or jumping dog, a dragon arriving, a trip to the World Cup—as possible continuations of the story. Those are suggestions, not additional demonstrated outcomes.

For creators, this feedback loop addresses a familiar complaint: the model can produce a compelling image, yet still resist the specific change the filmmaker wants. Ahres summarizes the appeal as “Instant feedback is the ultimate level of control.” The creator can observe the developing result and steer again, instead of relying entirely on the initial prompt.

Advertising follows from the same mechanism. Recent search interest could condition a logo inserted into a scene or an advertisement generated during viewing. Ahres is personally unenthusiastic about that application but expects it to grow. He also identifies an adoption barrier: brands care about exact visual identity, down to an incorrectly colored logo pixel. His eventual disappearance of preproduced ads is a forecast, contingent on overcoming that sensitivity.

4:244:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:24 · section reference included

Navigable worlds and live characters

The second category is a controllable world, exemplified by Google’s Genie 3. An image and, typically, text establish the environment; the user then controls a character within it. Games are the obvious application, but combining navigation with steerable video also creates experiences between games and movies. Ahres reports community projects exploring that combination. Netflix’s Bandersnatch, where viewers select the next scene, supplies a familiar analogy for audience agency.

Controllable simulation also suggests applications beyond entertainment:

  • Robotics: Vary an environment and the actions within it to generate training data. Ahres describes substantial activity among robotics labs and the prospect of effectively unlimited simulated examples.
  • Education: Place a learner inside a situation, such as a historical setting, rather than only describing it. This reflects his preference for experiential learning over a future centered on textbooks or LLM conversations.

The third category is the live interactive avatar. Ahres acknowledges that existing avatars, including those used for customer support, often feel strange or unconvincing. He expects emerging research-preview models to become more useful when combined with continuing video generation and controllable environments. Proposed applications include customer support, training, sales, gaming, and streaming services.

7:067:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:06 · section reference included

Applications built around audience input

An interactive livestream makes the control loop visible. Viewers propose what should happen next, vote, and influence the continuing video. Ahres reports Reactor users building this kind of experience, with X, YouTube, and Twitch as possible distribution surfaces. The application coordinates collective input; the model generates the next visible development.

The vote-selection step can be kept separate from model integration. This JavaScript function accepts a map from viewer IDs to their current choices, so each viewer contributes one active vote. It returns a proposed next prompt; sending that prompt to a live session is a separate operation.

javascript

function selectNextPrompt(votesByViewer) {
  const counts = new Map();

  for (const choice of votesByViewer.values()) {
    const prompt = choice.trim();
    if (!prompt) continue;
    counts.set(prompt, (counts.get(prompt) ?? 0) + 1);
  }

  const ranked = [...counts.entries()].sort(
    ([promptA, countA], [promptB, countB]) =>
      countB - countA || promptA.localeCompare(promptB)
  );

  return ranked[0]?.[0] ?? null;
}

Here, equal vote counts use alphabetical order as a simple application policy. The result remains a selection until the application submits it to generation.

Other reported projects apply the same input-and-response pattern to different activities:

  • Medical simulation: Add or remove a medicine and explore what happens next as a training playground for aspiring doctors. These are reported application concepts, not evidence of clinical validity.
  • Cooking simulation: Add an ingredient and inspect the generated result.
  • Video editing: Use video-to-video generation to add visual effects while editing through prompts, speech, or clicks.

Ahres says the editing platforms are not yet very good because model quality remains limited. Their significance is the interaction pattern: a user directs changes while seeing the developing output.

9:399:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:39 · section reference included

Four models behind the API

Reactor’s platform presentation turns those interaction patterns into four model choices:

ModelIntended workflow
HeliosInteractive video generation
LingBot-WorldNavigable generated worlds
LongLive 2Multiple shots forming a continuing film
SANA-StreamingLive video-to-video editing

Ahres attributes Helios to ByteDance and describes LingBot as Alibaba’s Genie-like model; the LingBot repository credits the Robbyant Team. He identifies LongLive 2 and SANA-Streaming as NVIDIA models. With LongLive 2, prompts can be prepared in advance to guide a consistent story across shots.

Four cards map Helios to interactive video, LingBot to navigable worlds, LongLive-2.0 to multi-shot film, and SANA-Streaming to live video editing.Open full source frame
Four models on one platform: Helios, LingBot, LongLive-2.0, and SANA-Streaming.

SANA-Streaming starts with existing footage rather than generating every element from scratch. A creator can upload recorded video or a Seedance 2 result, then add effects, remove people, or change the background. Ahres points to Hollywood previsualization as a useful setting for that workflow: explore a scene’s visual treatment before committing to its final production.

10:5911:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:59 · section reference included

A live session is not a batch job

Batch video generation can be simplified to a request, a cloud job, and a returned file. A live model needs infrastructure that keeps serving an evolving experience.

ConcernBatch generationLive generation
Unit of workRequest or jobContinuing session
DeliveryCompleted fileStreamed video
StateBounded generation taskContext retained across interaction

Streaming pixels from a server to a client adds delivery concerns that a file-returning workflow can avoid. Keeping the session running also makes memory a continuing responsibility.

Scene memory is a model limitation as well as a systems concern. Ahres recalls Genie 3 demonstrations where a character looks back and the world fails to preserve what was previously there. Turning left and then right should not casually replace the environment. Maintaining enough context to preserve that continuity remains difficult; a continuing stream alone does not solve it.

Geography adds another constraint. Ahres sets sub-100-millisecond latency worldwide as a requirement for the experience, not a measured Reactor service guarantee. A user in India or Japan should reach a GPU in that country or as close as possible. If compute is too distant, the delay interrupts the feedback loop that makes the medium interactive. Network latency, generated frame rate, and the time until an input visibly changes the scene are separate quantities; a single latency number cannot describe all three.

11:5111:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:51 · section reference included

Connecting an application to the stream

The developer-facing goal is to hide much of that infrastructure behind a small integration. Ahres estimates roughly ten lines of code, while acknowledging that the example is simplified. The basic sequence is to authenticate, load a model, and connect its output and controls to a video application, image workflow, or plugin. The slide shows the intended compact integration rather than the full operational responsibilities of an application.

Developer-first slide pairs an explanation of connecting and streaming a world with a short code example; the presenter points toward it.Open full source frame
A few calls to a living world.

For onboarding, the talk offers a signup QR code and the promotion code AIE2026 for $75 in credits. That is the offer made during the presentation; its continued availability is not established here. Ahres also characterizes the platform’s models as inexpensive, without supplying a cost comparison.

13:3513:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:35 · section reference included

Frame rate, broadcasting, and simulation rules

An audience member cites a stated 16 FPS for the game generator and asks what it would take to reach 30 FPS. Ahres names three approaches: multiple GPUs, model-weight optimization, and quantization. He frames the work as a matter of priorities, without reporting an achieved 30 FPS result.

A subsequent question suggests showcasing the technology at IBC, the International Broadcasting Convention in Amsterdam. Ahres says he was unfamiliar with the event and had not planned a showcase, but would look into it. The exchange connects the proposed medium to an existing industry concerned with producing and distributing video.

The next question asks about deterministic grounding: checking generated simulations against rule sets so that their behavior stays constrained. Ahres says Reactor does not provide those checks itself. He reports developers building such layers on the platform, open-sourcing them, and Reactor reusing community work. The division of responsibility is explicit: Reactor concentrates on the substantial infrastructure workload, while application builders can add rules that constrain a particular simulation.

14:2814:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:28 · section reference included

How do you know the world stayed consistent?

The final substantive question concerns measurement: how should visual consistency be evaluated? Ahres distinguishes fidelity, which he considers comparatively easy to assess, from the broader challenge of evaluating an interactive world over time. He acknowledges evaluations for real time, consistency, and fidelity, but describes overall evaluation as unsolved and current assessment as heavily dependent on looking at the result and applying human judgment.

That should not be read as an absence of quantitative evaluation. The LingBot-World paper reports VBench evaluation, including temporal and consistency measures, while acknowledging that world-model evaluation protocols remain immature. Existing video metrics and a solved general test for interactive worlds are different things. Ahres’s claim that nobody, including DeepMind, has solved the broader problem is his assessment of that research gap.

Asked what Reactor is doing about it, Ahres says its research team is working on the problem, without describing a method. The presentation therefore ends with an open engineering question: once users can act inside a generated stream, the system must be judged not only by how its frames look, but by whether the experience remains coherent as those actions accumulate.

16:2816:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:28 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…