▶ Watch ↗AI Engineer World's Fair 202514:25
Conference affiliation: CPO · Arize · 2026
A personally authored technical analysis of evaluation attributes, asynchronous traces, streamed outputs, and OpenInference instrumentation.
arize.comA co-authored account of managed agents, trace investigation, full-agent experimentation, adaptive judges, and production feedback loops.
arize.comA personally authored discussion of responsible machine learning and lessons from work on ML infrastructure at Uber.
arize.comArize's organization-owned open-source project for tracing, evaluation, experimentation, and debugging; not attributed as an individually authored repository.
github.comAparna Dhinakaran is the cofounder and chief product officer of Arize AI, where she develops tools for understanding AI behavior and turning failures into engineering improvements. Her work connects AI observability—recording the steps models and agents take—with evaluations that distinguish a wrong answer from the routing, retrieval, tool-use, or context failure that caused it.
Aparna studied electrical engineering and computer science at UC Berkeley, where her research included smart lighting and multi-vehicle collision avoidance. Internships exposed her to the work required to deploy an algorithm. At TubeMogul, developing a click-bot detection algorithm took roughly a month; deploying it occupied the rest of the summer.
Her early engineering career included machine learning at TubeMogul in 2014, software engineering in Apple’s Emerging Technologies group in 2015, and machine-learning engineering at Uber from 2016 to 2019. At Uber, she moved from Marketplace product teams into the team building MLaaS, later known as Michelangelo. Its model store and serving infrastructure had to support different model architectures and growing production demand. Troubleshooting required dashboards, scripts, and manual investigation.
Aparna subsequently entered Cornell’s computer-vision doctoral program, where she explored model bias, then created MonitorML and took it through Y Combinator. Arize acquired MonitorML, bringing her work together with Jason Lopatecki’s efforts to address similar production problems. They founded Arize in 2020.
Her 2021 writing on algorithmic accountability explains why model visibility matters beyond debugging. Excluding protected attributes does not eliminate bias when other variables correlate with them. Aparna argues that model builders need access to appropriate data to test unequal outcomes and must share responsibility for their systems’ consequences. Leaving that responsibility with users asks people to manage risks in systems they cannot inspect.
As AI development expanded into language models and agents, Arize developed tracing, evaluation, and experimentation tools for those systems. Arize Phoenix, launched in 2023, provides a self-hosted environment for inspecting traces, assembling datasets, and comparing changes to prompts, models, and retrieval. Arize AX provides managed workflows across development and production. Aparna leads product work and contributes technical writing and experiments; Phoenix and AX are team-built products.
Aparna and Jason jointly announced Arize’s $70 million Series C in February 2025. Dynatrace completed its acquisition of Arize on October 1, 2026, welcoming Aparna as Arize’s chief product officer. The companies plan to connect AI evaluation with observability across the surrounding software and infrastructure, while Arize AX and Dynatrace remain standalone offerings and Phoenix remains available as an open-source project.
Aparna’s evaluation work separates behaviors that an overall score can obscure:
Arize’s assistant Alyx exposed the limits of predefined evaluations as it gained longer memory and the ability to generate interfaces dynamically. It could lose context, fail to recognize completion, or repeatedly call the same tool. Those experiences shaped Aparna’s case for adaptive agent judges. She proposes using them alongside deterministic tests and conventional model judges: fixed checks cover known requirements, while an investigating agent can search changing trajectories for failure patterns the team has not anticipated.
Her June 2026 article with Jason connects that investigation to engineering changes. Signal reviews production traces and groups emerging problems into investigations. Repository-aware managed agents gather context and propose repairs for human review. Full-agent experiments test whether those repairs help without introducing regressions elsewhere; a model upgrade might improve final answers while making tool calls less reliable. Voice agents add audio, interruptions, transcription quality, and speech latency to the behaviors being examined. Engineers review and approve proposed actions.
Aparna and Prukalpa Sankar’s work on context and agent harnesses distinguishes another pair of failure sources. Business definitions and policies determine what an agent knows; its harness governs how it acts through tools, memory, permissions, and handoffs. A capable model can reason coherently from stale information and still produce the wrong result. Evaluating those layers separately helps distinguish a knowledge problem from an execution problem.
The feedback process depends on instrumentation that captures enough information to explain a failure. Aparna’s OpenTelemetry analysis, acknowledging contributions from Roger Yang, Xander Song, and John Gilhuly, describes practical mismatches between standard tracing and AI workloads. Evaluation results can arrive after a span has ended, streaming output requires keeping spans open until completion, and attribute limits can discard fields an evaluator needs.
For late evaluation data, she describes augmenting and materializing spans in Arize, while Phoenix joins a separate metadata table to spans. The join approach works for smaller datasets but adds computational overhead at larger volumes. OpenInference extends OpenTelemetry instrumentation to capture AI-specific behavior across frameworks.
In a public post crediting Phoenix’s open-source lead Mikyo King and his small team, Aparna argued that exceptional engineers build systems whose benefits compound. Preserving failures as test cases and diagnoses as reusable instructions gives that position a concrete expression in her work.
▶ Watch ↗AI Engineer World's Fair 202514:25
▶ Watch ↗AI Engineer Summit 202515:28
▶ Watch ↗AI Engineer World's Fair 20266:06
▶ Watch ↗AI Engineer Code 202510:56
Loading more talks and highlights…