▶ Watch ↗AI Engineer World's Fair 20255:21
Conference affiliation: Dev Rel Lead · Warp · 2026
Ben Holmes’s whiteboard videos on web development.
bholmes.devBen Holmes is a developer-experience engineering lead at Warp, an Astro core maintainer, and the creator of Hubble, an open-source Markdown notebook for people and coding agents. His contributions range from typed content tools in Astro to interfaces that combine voice and text, and agent-evaluation workflows that uncover mistakes missing from an assistant’s final report.
At Astro, Ben coauthored the 2022 release announcement introducing integrations and the astro add command. In 2023, he introduced Astro Content Collections, which let authors define schemas for Markdown and MDX entries, catch malformed frontmatter, and query validated content with generated TypeScript types. A missing field or unexpected value could produce an actionable development error rather than remain hidden until someone encountered a broken page. He also coauthored the 2024 release announcement that made Astro Actions stable, supporting type-safe backend functions and form handling.
Ben also teaches through whiteboard explanations and small implementations of larger systems. simple-rsc reconstructs React Server Components, simple-sync-engine explores SQLite-backed synchronization, and simple-stack pares down web-application architecture. These projects give developers code they can inspect alongside the concepts he explains.
In his 2025 FixGPT prototype, Ben addressed two frustrations with ChatGPT: voice conversations made written material awkward to retrieve, and model menus required users to understand product names before asking a question. An assistant could read an email draft aloud, but collaborating on the written version meant ending the call and finding the transcript.
FixGPT combined realtime audio with a persistent chat panel, borrowing the familiar experience of texting someone during a call. A tool call let the assistant send links, drafts, and longer explanations in writing while the voice conversation continued. Another tool could hand a demanding question to a reasoning model. Ben described heuristic routing—such as recognizing a request for depth or pros and cons—as a way to choose additional reasoning on the user’s behalf. The prototype made his interface argument concrete: users could ask for the help they wanted without first choosing a model or switching conversation modes.
Hubble applies a similar concern for reducing effort to personal notes. It keeps knowledge in ordinary Markdown files that their owner and coding agents can both read. Ben favors quick capture, often through voice dictation, followed by agent-assisted organization. The initial note can be rough; the enrichment workflow supplies structure afterward.
His knowledge-base demonstration explains several decisions that keep this process manageable:
Ben extends these connected notes into topic wikis, people indexes, and clickable HTML visualizations, crediting Andrej Karpathy for the wiki-generation idea. He also demonstrates scheduled processing: the Obsidian CLI synchronizes Markdown into a Docker-backed cloud sandbox, an agent enriches notes or updates a wiki, and the changed files sync back. He prefers this background synchronization to manually pushing and pulling a notes repository. The notebook remains the place to capture and read; the agent handles recurring organization.
On Warp Factories, Ben and his team encountered agents that worked around stale testing instructions without mentioning the detour in their final reports. The agents repeatedly probed ports to understand their environment before starting useful work. To engineers, the problem initially appeared as an unexplained delay; the tool history exposed what needed fixing.
In his account of self-improving agents, Ben argues for evaluating three things together: the conversation, the tool calls, and the resulting work. His team uses scoring rubrics for wasted effort, redundant tests, and excessive verbosity. These criteria reflect practical costs: unnecessary exploration consumes time, while excessive comments and summaries make code review harder. Test scoring follows the team’s stated testing priorities rather than treating more tests as an automatic improvement.
A scheduled agent can then examine failing reports, identify recurring problems, and propose changes to skills or configuration. Ben demonstrated Skill Doctor as a way to apply this analysis to local coding-agent histories. Its suggestions included clearer instructions for finding an available port and more specific changelog guidance to reduce repeated editing. In his team’s factory setup, another proposed correction supplied missing configuration that had prevented a triage agent from spawning subagents.
Ben also distinguishes proposing an improvement from knowing whether it helped. He advocates tracking scores over time and comparing changes in those scores with changes to the agent setup. Unlike an invisible overnight memory update, this gives the team a view of what the agent proposed, what people merged, and whether subsequent runs improved.
▶ Watch ↗AI Engineer World's Fair 20255:21
Loading more talks and highlights…