← All speakers

Laurie Voss

Conference affiliation: Arize · 2026

  • Ship Real Agents: companion slides ↗

    Slides explicitly identified as the companion material for Voss’s workshop, including setup instructions, a learner-notebook link, evaluator examples, and experiments.

    slides.com · Source ↗
  • LLM Evaluation Basics ↗

    Arize University course explicitly led by Voss, covering tracing, deterministic checks, LLM judges, clear criteria, and incremental evaluation.

    courses.arize.com · Source ↗
  • Arize AX | How to Observe, Evaluate, and Improve AI Agents ↗

    Hands-on course series attributed to Voss, using a financial-analysis agent to teach tracing, evaluation, calibration, experiments, monitoring, and feedback loops.

    courses.arize.com · Source ↗
  • stuff-everybody-knows ↗

    Repository under Voss’s verified GitHub account, described as a guide to the rest of a web-development career. Retained profile evidence establishes account ownership, but not sole authorship or the biography’s teaching start date.

    github.com · Source ↗
  • domain-name-generator ↗

    Repository under Voss’s verified account for a web application combining random word generation with the Namecheap API to find domain names; sole authorship is not established.

    github.com · Source ↗
  • zeitgeist ↗

    Repository under Voss’s verified account, described as a small application that summarizes a social-media feed; sole authorship is not established.

    github.com · Source ↗

Laurie Voss is the co-founder of npm, Inc. and the head of developer relations at Arize AI. After helping build infrastructure for the JavaScript ecosystem, he now concentrates on making AI agents observable, testable, and reliable.

On this page

From web development to AI infrastructure

Voss identifies himself as a web developer on his GitHub profile and as an npm, Inc. co-founder on his personal site. In his evaluation workshop, he describes his transition from speaking about JavaScript to thinking about how to test AI systems and make them work.

His verified GitHub account includes Stuff Everybody Knows Except You, described as a guide to the rest of a web-development career. His personal site discusses exploring AI, ML, and LLM opportunities after leaving Netlify. His retained conference recording credits also identify a LlamaIndex affiliation before his Arize presentations. At Arize AI, his evaluation teaching focuses on tracing, deterministic checks, LLM judges, calibration, and measured improvements to agents.

How Voss thinks about dependable AI

  • Agents should solve concrete software problems. His hands-on evaluation workshop uses a financial-analysis agent that researches companies and writes reports. He distinguishes a report that merely summarizes financial information from one that supports an investment decision through forward-looking analysis and an actionable recommendation. Tests should enforce the required outcome and meaningful constraints without prescribing incidental tool sequences, since an agent may find a faster valid route.
  • Answers need the right evidence. The workshop separates research from report writing so that an evaluator can compare the final answer with the research supplied. Voss prioritizes fixing stale knowledge, wrong sources, and missing evidence: prompt engineering cannot recover facts the system never receives.
  • Agent evaluation starts with understanding failure. Voss combines execution traces, inexpensive deterministic checks, human-labeled examples, and LLM-as-a-judge rubrics. His hands-on evaluation workshop demonstrates why a superficially plausible financial report can still concern the wrong company, omit an investment recommendation, or be delivered incorrectly.
  • Evaluators need their own safeguards. Generic correctness checks can fail when a judging model lacks current information; a faithfulness evaluation instead tests whether an answer reflects the research supplied. Voss recommends checking automated judgments against labeled examples, watching for bias, and avoiding tests so prescriptive that they reject better solutions.
  • Capability and regression evaluations serve different purposes. Capability tests define behaviors an agent has yet to master; regression tests preserve behaviors that already work. Voss prioritizes better data before prompt changes, model selection, or parameter tuning, and regards fully automated optimization as promising but immature.

Voss also treats developer relations as engineering and product work, emphasizing documentation, useful open-source software, education, and actionable product feedback. His writing examines how AI-assisted development can weaken the junior-engineer apprenticeship pipeline and how autonomous agents complicate legal accountability for security breaches.

Read the topics behind these talks

8 conference talks

Loading more talks and highlights…

References