AI Engineer World's Fair 2026

HTML Is All Agents Need

Loading slides…

Read the talk

HTML Is All Agents Need

HyperFrames turns familiar web code into video by pairing a thin authoring format with controlled browser rendering, while leaving room for craft and human editing.

From a talk by James Russo

Before you start: Basic familiarity with HTML, CSS, and JavaScript will help with the animation example; no video-rendering experience is required.

The product is built. Now make the launch video.

The recording opens with a music video: a finished piece of generative media before any explanation of the tooling. James Russo says an agent made it completely in a single shot, using HTML. Introducing himself as the co-creator and tech lead of HyperFrames at HeyGen, he asks a practical question: who now finds making the launch post or launch video harder than building the product with a coding agent? Some hands go up. Software creation has become easier; communicating what that software does still takes work.

HeyGen approaches that gap through its mission of communication through video. Its avatar provides A-roll: the narrator, character, or main subject. But a speaking subject alone makes a fairly plain video. B-roll supplies images and other media; animation, captions, and music add the remaining layers. Giving an agent a useful video canvas means letting it compose all of these pieces, not merely generate the person speaking.

Slide comparing a man speaking into a microphone with a woman in a video frame containing text overlays.Open full source frame
A-roll beside a video overlay example labeled “B-roll + polish.”
0:360:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:36 · section reference included

Use the language the model already knows

HTML is the proposed canvas, with CSS for styling and JavaScript for behavior. Russo’s rationale is familiarity: models have substantial exposure to web code, so these languages give them an expressive starting point. He calls them the models’ native languages. A custom DSL or JSON structure adds a translation task before the agent can make anything visually interesting. His analogy is asking Shakespeare to write a poem in Japanese or Chinese: examples can teach the format without necessarily preserving the fluency of the original language.

This interest in HTML extends beyond video. Russo shows posts by Thariq and Andrej Karpathy discussing HTML as the new Markdown—a way for models to produce visual representations instead of only text. He says his talk submission preceded both posts. The useful design question is whether a familiar, expressive output format lets the model spend more of its effort on the result rather than on learning how to represent it.

Slide headed “The field is converging on HTML,” with posts by Thariq and Andrej Karpathy side by side.Open full source frame
Two posts discuss HTML as an output format for LLMs.
2:132:23
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:13 · section reference included

Quality is only half the authoring problem

The team arrived at HTML after more than a year of experiments. Russo compares the alternatives along two axes: quality of output and friendliness to agents. His account distinguishes tools that help a knowledgeable human operate an editor from formats in which an agent can author a composition directly.

ApproachWhat worked for the teamWhere it became difficult
After Effects and Premiere ProHigh-quality creative outputConnectors mainly assisted existing editor expertise
Lottie and RivePromising visual resultsAgent familiarity, editing, and control
RemotionStrong examples of video through codeFramework instruction and examples consumed attention

These are Russo’s observations from the team’s experiments. He groups Lottie and Rive with specialized JSON or XML representations, but Rive’s runtime format is actually binary .riv; its JSON core definitions are a separate implementation detail. The broader concern was how much representation-specific instruction the agent needed. With Remotion, Russo says teaching the framework and supplying examples ultimately constrained the creativity they were seeking.

The next change came with Gemini 3 and other newer models, around November 2025. Russo reports a step improvement in the team’s results: when given examples of the desired output, models naturally gravitated toward HTML, CSS, and JavaScript. That suggested a different engineering strategy—support the representation the models were already choosing.

3:343:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:34 · section reference included

The thinnest workable HTML contract

The team used Gemini 3 Flash as a design partner. Flash followed the initial Gemini 3 introduction in December 2025. Their heuristic was to make the format workable for a smaller model, expecting more capable coding agents to handle it too. They tried wrappers around web code, more context, larger system prompts, and skills. The thinnest wrapper won: ordinary HTML with a few data-* attributes carrying metadata such as timing.

The demonstration places HTML on the left and its rendered webpage on the right. The browser handles both preview and final rendering, so the composition does not need to be translated into a separate visual language for export. The same HTML becomes the preview and the source of the video frames. HyperFrames packages that approach as an open-source framework for turning agent-authored HTML into video.

5:245:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:24 · section reference included

Stop the browser clock before capturing the frame

Authoring the composition is only the first problem. Producing a deterministic MP4 is harder because browsers are deliberately asynchronous. A page can initially display fallback text, then change its typography when a font arrives. Images and videos can appear later still. Those behaviors are reasonable for an interactive page on a variable network; they are undesirable when a captured frame permanently records the incomplete state. Video needs every intended asset ready at capture time.

HyperFrames treats video as a sequence of still images and controls rendering accordingly:

  1. Freeze the browser’s clock.
  2. Seek the composition to the time of the next frame.
  3. Wait for the required assets and verify that the page is ready.
  4. Capture a screenshot.
  5. Advance to the next frame and repeat.
  6. Encode the collected frames into the video.

The renderer advances composition time explicitly instead of recording whatever the browser happens to display while wall-clock time passes.

A small browser example makes the seek operation concrete. This title moves across the screen over two seconds; setting currentTime selects its state without waiting for playback:

html

<!doctype html>
<meta charset="utf-8">
<style>
  body { margin: 0; background: #101828; }
  h1 { color: white; font: bold 64px sans-serif; }
</style>
<h1 id="title">Ready to launch</h1>
<script>
  const title = document.querySelector('#title');
  const motion = title.animate(
    [
      { transform: 'translateX(0px)' },
      { transform: 'translateX(240px)' }
    ],
    { duration: 2000, fill: 'both', easing: 'linear' }
  );
  motion.pause();

  window.seek = (seconds) => {
    motion.currentTime = seconds * 1000;
  };
  window.seek(1);
</script>

At one second, the title is translated 120 pixels. This illustrates the animation side of the mechanism; asset readiness and screenshot capture belong to the renderer. Russo connects the HyperFrames name to using the same browser input for preview and video. That shared input reduces discrepancies, but reproducibility also requires controlled fonts, assets, randomness, and browser environment. His rendering companion article distinguishes controlled Linux capture from less robust screenshot heuristics on macOS and Windows.

7:037:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:03 · section reference included

The browser becomes the visual toolbox

Using the browser opens up an existing ecosystem: Three.js, charts, SVGs, shaders, WebGL, WebGPU, and Lottie can all supply visual material. There is an additional requirement for animation, however. Content that owns its own clock needs a seekable adapter or prerendered frames so the renderer can request the right state at the right time. Browser compatibility supplies the pixels; controllable time makes those pixels usable in deterministic video.

That changes the creative workflow too. The team looks for inspiration on the internet, takes useful examples, adapts them to the composition, and incorporates them into videos. A browser effect can become a video element without first being rebuilt inside a conventional motion graphics editor.

8:569:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:56 · section reference included

Teach motion, pacing, and taste

A rendering framework makes a video possible; it does not make the video good. HyperFrames couples the framework with skills focused on taste and video principles. Because the agent already knows web languages, the instructions can concentrate on motion, pacing, and craft instead of spending most of their space teaching framework syntax. The team repeatedly evaluates output and uses agents to improve those skills, aiming to raise the quality floor of a single prompt.

Three connected boxes show great videos, skills encoding motion, pacing and craft, and refinement through evals and agents.Open full source frame
Skills encode motion, pacing, and craft from great videos.

The website-to-video demonstration starts with only a website. The skill directs the agent to visit it, retrieve relevant assets and information, and capture its design, theme, and branding in Design.md, or Frame.md in HyperFrames’ workflow. Basic motion examples then guide how that material becomes an animated composition. The website supplies the visual identity; the skill supplies a process for turning it into a video.

9:259:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:25 · section reference included

From a usable first pass to a finished video

Single-prompt output can be sufficient, but polished work still requires decomposition and iteration. Russo compares this with AI-assisted software engineering: having something that works does not eliminate the need for judgment. HyperFrames preserves the production process the team used before the framework rather than replacing it with a single generation step.

Their sequence moves from intent to composition to editing:

  1. Define the narrative, vision, and mission of the video.
  2. Storyboard it frame by frame, deciding what each frame needs to accomplish.
  3. Add motion in HTML, CSS, and JavaScript.
  4. Merge the pieces into one cohesive video.
  5. Use HyperFrames Studio for the final manual adjustments, including dragging and tweaking elements.

The editor keeps humans involved after code generation, when a small visual adjustment may be easier to make directly than to describe in another prompt.

Russo then demonstrates keyframes, which he says were released that day. They let a creator coordinate different motions frame by frame or keyframe by keyframe in Studio. He positions those controls against the work a professional motion designer would do in After Effects: the goal is to retain detailed creative control alongside agent authoring.

10:5411:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:54 · section reference included

Use the coding agent already building the product

Russo says HyperFrames had launched a couple of months earlier and was already operating at what the team considered scale. He reports over 1.3 million videos rendered by open-source users in the preceding 90 days. He also reports 267,000 creators had tried it, about 15,000 videos were being rendered daily with the open-source framework, and the repository had 32,000 GitHub stars. These are usage figures reported at the talk, rather than measurements of rendering speed or creative quality.

The framework is open source under Apache-2.0, and Russo describes it as free forever. His promise concerns the project; it should not be read as a guarantee that every hosted service or generated media asset has no cost. He names Claude Code, Codex, and Cursor as compatible agents, with the broader requirement being the ability to write HTML, CSS, and JavaScript. That brings the workflow back to the opening problem: the agent helping build the product can also help make its launch video.

12:3512:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:35 · section reference included

Creative quality remains an open problem

Russo’s closing qualification is direct: models still struggle with creative work. The team continues evaluating and improving skills, but it sees a need for progress beyond the framework itself. It has begun work on a code-to-video benchmark, seeking collaboration with model labs and people building video agents to raise the quality floor for everyone. This is work underway, not a presentation of benchmark results. Russo invites interested collaborators to speak with him or the HyperFrames team at the event.

13:2813:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:28 · section reference included

A showcase, then the project handoff

The final demonstration is the AI Engineer World’s Fair showcase video, created with HyperFrames through substantial collaboration with the event team. It presents motion graphics and design built with HTML, CSS, and JavaScript, complementing the opening single-shot example with a collaboratively produced piece.

The closing returns to the gap between building and launching. Familiar web code gives the coding agent a route into video production; rendering infrastructure, skills, and editing make that route useful. Russo ends by pointing viewers to the project and open-source repository, inviting them to download the skills and try the workflow on the product they are already building.

Closing slide with the HyperFrames logo, “HTML is all agents need,” a QR code, repository address, and X contact.Open full source frame
HyperFrames closes with “HTML is all agents need” and a project QR code.
14:0914:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:09 · section reference included

Resources

From the talk

  • Open-source HTML-to-video framework with installation instructions, coding-agent skills and seekable animation workflows.

  • Technical specification for Rive's binary runtime format, including versioning and serialized objects.

  • Google's December 2025 announcement of the Flash model used in HyperFrames' authoring experiments.

Transcript download
Full transcript

Loading transcript…