Fuad Ali
Conference affiliation: Senior Product Manager · Arize · 2026
Related work and projects
Fuad Ali is a Staff AI Product Manager at Arize AI whose work connects agent observability, evaluation, and experimentation. He builds workflows for turning failures into testable changes, while examining a problem that automatic improvement can obscure: a change may fix one mistake and damage a decision the agent already got right.
On this page
From systems engineering to product development
Fuad studied engineering at McMaster University and worked on Tesla’s mobile application and energy software, flight software at SpaceX, and distributed content-moderation systems at Twitter. At Federato, which he joined in 2023, he moved from senior engineering into product leadership. His work expanded from delivering an insurance platform to interviewing users, defining product requirements, and directing a homepage redesign.
He joined Arize in June 2025 and became a staff product manager in September 2026. His responsibilities span product strategy, self-service adoption, and hands-on development, including work on the evaluation and post-training loop for Alyx, Arize’s in-product agent.
Turning failures into tested improvements
Fuad’s product work treats traces as inputs to investigation and experimentation. Recording what happened is the starting point; the next questions are what caused the failure and whether a proposed change improves behavior across relevant tasks.
- Regression tests must include successes. In work with Nancy Chauhan on agent regression testing, Fuad examined a shopping agent that cancelled an order without obtaining the necessary details or confirmation. Added safeguards made cancellation safer, but also stopped a previously successful purchase from completing. In the demonstration, average task completion fell from 0.89 to 0.72. Comparing both configurations on the same dataset—including successful workflows alongside failures—and measuring action safety separately from completion exposed the tradeoff. A safer cancellation policy could still leave a customer unable to buy.
- Coding traces make workflow choices testable. With Duncan McKinnon and Chris Cooning, Fuad co-authored the introduction to coding-harness-tracing. The project captures file operations, commands, tool calls, retries, latency, and token usage across coding assistants. These records help developers investigate whether an agent read relevant files, repeated unnecessary work, or ran tests after editing. They also support comparisons of prompts, models, tool configurations, and reusable skills across real tasks.
- Access and procedural knowledge serve different needs. In joint writing with Dirk Brand and Sragvi Vadali, Fuad distinguishes MCP, which exposes operations inside compatible agent clients, from a CLI for scripts and shell workflows, and skills that supply procedures and practical cautions. An experiment-comparison skill, for example, can teach an agent to resolve the correct project and inspect runs in the right order. The useful interface depends on the user’s environment and task.
His self-learning loops workshop connects these ideas through a shopping assistant that returns products outside a user’s budget because its price filter is broken. An agent uses traces alongside the application repository to investigate the defect and propose a fix. The validation step then replays failure cases against a development endpoint and compares quality, latency, and cost before a human reviews the proposal.
The example also sharpens his approach to evaluation: a judge may need access to tool calls, code, and external context to assess an outcome. Checking whether returned prices satisfy the requested range gives the evaluator a concrete condition to test, beyond judging whether the answer sounds plausible.
When should one agent overrule another?
Fuad’s independent research examines multi-agent decision-making. Earlier simulations explored how legislative institutions translate people’s preferences into policy. His subsequent research on reviewer authority separates two questions: whether a reviewer gives good advice, and whether that advice should be allowed to replace an existing decision.
Using synthetic tasks with known correct answers, he compared broad override powers with evidence-gated review, where a reviewer had to demonstrate a checkable violation before replacing a proposal. Restricting intervention protected correct decisions, but also blocked some improvements to proposals that were permissible yet suboptimal. He calls this tradeoff the correction-corruption frontier: broader intervention can repair more errors while giving mistaken reviewers more opportunities to damage good answers.
Fuad also identified a problem in his initial experimental interface. Asking reviewers to recommend a policy could encourage unnecessary changes. Adding explicit options to retain, replace, or escalate produced different behavior in Qwen and Mistral. The interface affected the results without solving the problem uniformly. These experiments use mechanically checkable objectives; they do not establish that the same safeguards will resolve ambiguous real-world decisions.
A separate preference-following experiment showed why aggregate scores can conceal unstable decisions. Reversing the order of options changed 16 of 120 answers while barely moving overall accuracy: some errors disappeared as new ones appeared. Because the task’s preference rules were already explicit, Fuad favored ordinary code for that task.
Outside his product and research work, Fuad co-hosts The Next Iteration with Damien Joseph. Established in 2020 and formerly called Into the Adultverse, the podcast explores careers, technology, health, and personal development through conversations with builders, physicians, founders, and researchers.
Read the topics behind these talks
5 conference talks
▶ Watch ↗AI Engineer World's Fair 202638:55
Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
Explore the talk →
▶ Watch ↗17:59
The Transcript Looked Fine. The Call Wasn't. — Debugging Voice Agents, Arize
Explore the talk →
▶ Watch ↗20:06
We Kept Rebuilding Our AI Agent. Here's What Survived - Fuad Ali, Arize AI
Explore the talk →Loading more talks and highlights…

