Laurie Voss

Laurie Voss is the co-founder of npm, Inc. and the head of developer relations at Arize AI. After helping build infrastructure for the JavaScript ecosystem, he now concentrates on making AI agents observable, testable, and reliable.

From web development to AI infrastructure

Voss identifies himself as a web developer on his GitHub profile and as an npm, Inc. co-founder on his personal site. In his evaluation workshop, he describes his transition from speaking about JavaScript to thinking about how to test AI systems and make them work.

His verified GitHub account includes Stuff Everybody Knows Except You, described as a guide to the rest of a web-development career. His personal site discusses exploring AI, ML, and LLM opportunities after leaving Netlify. His retained conference recording credits also identify a LlamaIndex affiliation before his Arize presentations. At Arize AI, his evaluation teaching focuses on tracing, deterministic checks, LLM judges, calibration, and measured improvements to agents.

How Voss thinks about dependable AI

• Agents should solve concrete software problems. His hands-on evaluation workshop uses a financial-analysis agent that researches companies and writes reports. He distinguishes a report that merely summarizes financial information from one that supports an investment decision through forward-looking analysis and an actionable recommendation. Tests should enforce the required outcome and meaningful constraints without prescribing incidental tool sequences, since an agent may find a faster valid route.

• Answers need the right evidence. The workshop separates research from report writing so that an evaluator can compare the final answer with the research supplied. Voss prioritizes fixing stale knowledge, wrong sources, and missing evidence: prompt engineering cannot recover facts the system never receives.

• Agent evaluation starts with understanding failure. Voss combines execution traces, inexpensive deterministic checks, human-labeled examples, and LLM-as-a-judge rubrics. His hands-on evaluation workshop demonstrates why a superficially plausible financial report can still concern the wrong company, omit an investment recommendation, or be delivered incorrectly.

• Evaluators need their own safeguards. Generic correctness checks can fail when a judging model lacks current information; a faithfulness evaluation instead tests whether an answer reflects the research supplied. Voss recommends checking automated judgments against labeled examples, watching for bias, and avoiding tests so prescriptive that they reject better solutions.

• Capability and regression evaluations serve different purposes. Capability tests define behaviors an agent has yet to master; regression tests preserve behaviors that already work. Voss prioritizes better data before prompt changes, model selection, or parameter tuning, and regards fully automated optimization as promising but immature.

Voss also treats developer relations as engineering and product work, emphasizing documentation, useful open-source software, education, and actionable product feedback. His writing examines how AI-assisted development can weaken the junior-engineer apprenticeship pipeline and how autonomous agents complicate legal accountability for security breaches.