Ian Webster is the co-founder and chief executive of Promptfoo, the open-source AI security and evaluation company now part of OpenAI. He builds tools that help developers identify dangerous behavior in language-model applications before those systems reach customers or gain access to sensitive information.
Webster previously held engineering roles at Room 77 and Google and created Asterank, a database for visualizing and assessing asteroids that Planetary Resources acquired in 2013. Its visualization technology later informed SpaceKit, an interactive space-visualization library. His other projects include Ancient Earth, which maps the planet’s geological history. He also co-founded Zenysis, serving as chief technology officer of a company building data-integration software for governments confronting public-health emergencies and humanitarian crises.
At Discord, Webster founded and led developer-platform and developer-relations teams before helping build Clyde, the company’s conversational AI assistant. Deploying an AI product at consumer-platform scale made safety, privacy, legal exposure, and adversarial misuse immediate engineering problems. Webster began developing Promptfoo’s open-source evaluation framework during that work; its first public release arrived in 2023, and he founded the company with Michael D’Angelo in 2024.
Developer-first evaluation: AI evaluations should behave like software tests: local, fast, understandable, and integrated into pull requests. Webster favors targeted deterministic checks, including testing whether informal chatbot responses begin with lowercase letters and assessing retrieval triggers separately from summarization.
Application-specific red teaming: Safe base models can become vulnerable once developers add system prompts, retrieved documents, tools, or external integrations. Webster tests complete applications for jailbreaks, harmful outputs, sensitive-data leakage, and unintended actions before deployment.
Model-specific prompt design: Accumulating special-case instructions can degrade performance, and prompts tuned for one model family may fail on another. Webster advocates simpler prompts and fresh evaluation when changing providers.
Privacy-aware production feedback: At Discord, his teams built evaluation cases from internal testing and public reports without recycling private conversations, kept prompts in Git, and reused existing observability infrastructure. They also experimented with occasional stronger-model responses to correct conversational personality drift.
Discord’s experience with Clyde shows how small evaluations, familiar development tools, and deliberate adversarial testing can make LLM risks easier to measure before launch.
▶ Watch ↗