Dat Ngo is an AI architect and director of solutions for Europe, the Middle East, and Africa at Arize AI, helping enterprises make AI agents and machine-learning systems observable, measurable, and reliable in production. His work combines applied statistics, practical evaluation design, and operational safeguards for systems whose behavior can change from one execution to the next.
From statistical modeling to AI infrastructure
Ngo holds a master’s degree in applied statistics from Texas A&M University and previously held data-science leadership roles at Wood Mackenzie, alliantgroup, and PointPredictive. At Arize, his work initially included fraud-model monitoring and drift detection, examining how production data diverges from training conditions and how those changes affect fraud prevention.
He subsequently extended those concerns to generative AI, working across Arize AX and Arize Phoenix, its open-source observability and evaluation platform. His work with Phoenix, Langflow, and NVIDIA demonstrates how teams can benchmark retrieval configurations and improve application quality through structured experimentation. His analysis of AI infrastructure in banking addresses the additional complications of distributed business units, governance requirements, and regulated environments.
How Ngo approaches reliable AI
OpenTelemetry-based agent observability: Traces and spans reveal which tools an agent called, where latency accumulated, and whether dependent steps occurred in the proper order. Comparing many executions exposes recurring failure patterns that an isolated successful answer can conceal.
Agent trajectory evaluation: Ngo evaluates individual model calls, interactions between components, complete tool-use sequences, and extended conversations separately. A convincing final response does not establish that an agent followed the correct process or satisfied its user, as he explains in his AI Engineer session on agent observability.
Calibrated, cost-aware evaluation: Human-labeled golden datasets help verify whether an LLM judge approximates trusted expert judgment. Schema validation, keyword checks, and other deterministic tests can answer narrower questions without adding unnecessary model cost or latency. Ngo also treats evaluators as systems requiring continuous correction, not fixed measures of truth.
Prompts as runtime configuration: When prompts change independently of application code, Ngo advocates versioning, validation, controlled rollout, rollback, and execution traceability. Model settings, tools, and retrieval behavior may belong in the same operational lifecycle; authorization rules and core application guarantees do not.
Ngo’s longer-term focus is automated observability and evaluation: giving coding agents programmatic access to traces, diagnostics, and experiments so teams can identify failures, refine evaluators, and improve AI applications without manually inspecting every dashboard.
Scalable evaluation combines traces, human labels, model judgments and code checks, then uses that signal to improve applications, diagnose agent paths and refine the evaluators themselves.
▶ Watch ↗