← All speakers

Bio, Work & Ideas

Wyatt Marshall

Conference affiliation: Halluminate · 2025

Wyatt Marshall is the co-founder and chief technology officer of Halluminate, which develops benchmarks, training environments, and evaluation infrastructure for AI agents that operate browsers and other software. His work exposes the difference between agents that can retrieve information from websites and those that can reliably complete complex, consequential actions.

Marshall studied computer science and philosophy at Cornell University, where he participated in the Milstein Program and met future co-founder Jerry Wu. Before founding Halluminate, he worked in software and data engineering at early-stage New York startups. At MediaWallah, which he joined in December 2022, he maintained production data infrastructure and built Python tooling for ETL workflows and AWS automation, as he described in an interview about his engineering work.

Marshall and Wu founded Halluminate in 2024 and joined Y Combinator’s Summer 2025 batch. Marshall also published the company’s Python API package.

  • Real-world browser-agent evaluation. Halluminate and Skyvern created WebBench, an open benchmark with 2,454 public tasks across 452 live websites. Its key distinction separates retrieving information from actions that create, update, or delete data.
  • Why action-taking agents fail. Write tasks demand longer sequences, more complicated interfaces, dynamic forms, and authentication. In his AI Engineer World’s Fair presentation, Marshall distinguished model errors from infrastructure failures involving CAPTCHAs, blocked proxies, inaccessible verification emails, and anti-bot defenses.
  • Browser infrastructure and latency. Halluminate’s BrowserBench evaluates browser providers and execution reliability. Marshall emphasizes that repeated observation, reasoning, and interaction make agents particularly difficult to deploy in real-time applications, even when asynchronous workflows remain practical.
  • Training environments for computer-use agents. Halluminate develops managed software sandboxes where agents can practice realistic tasks without the unpredictability and unintended consequences of operating directly on production websites.

Read the topics behind these talks

1 conference talk

Key ideas

Scroll to read ↓

WebBench exposes the gap between retrieving information and completing state-changing browser workflows, where authentication, infrastructure, latency, and unpredictable agent behavior become decisive.

  • Buying a power tool requires an execution loop
    0:01 ↗
  • Evaluating agents means evaluating tasks, judges, and infrastructure
    3:28 ↗
  • Information retrieval works surprisingly well
    5:49 ↗
  • Writing introduces longer trajectories and hostile boundaries
    7:01 ↗
  • Separate agent mistakes from infrastructure failures
    10:44 ↗
  • The observe-decide-act loop makes latency a product constraint
    13:10 ↗
  • Choose the task category before choosing an agent
    14:17 ↗
  • Combine deterministic tracks with agent-driven exploration
    16:52 ↗
  • Longer memory, authentication, and payments remain open problems
    17:54 ↗
  • Agents already produce surprising real-world side effects
    19:10 ↗

References