← All speakers

Aparna Dhinakaran

Conference affiliation: CPO · Arize · 2026

Aparna Dhinakaran is the cofounder and chief product officer of Arize AI, where she develops tools for understanding AI behavior and turning failures into engineering improvements. Her work connects AI observability—recording the steps models and agents take—with evaluations that distinguish a wrong answer from the routing, retrieval, tool-use, or context failure that caused it.

On this page

Learning what production demands

Aparna studied electrical engineering and computer science at UC Berkeley, where her research included smart lighting and multi-vehicle collision avoidance. Internships exposed her to the work required to deploy an algorithm. At TubeMogul, developing a click-bot detection algorithm took roughly a month; deploying it occupied the rest of the summer.

Her early engineering career included machine learning at TubeMogul in 2014, software engineering in Apple’s Emerging Technologies group in 2015, and machine-learning engineering at Uber from 2016 to 2019. At Uber, she moved from Marketplace product teams into the team building MLaaS, later known as Michelangelo. Its model store and serving infrastructure had to support different model architectures and growing production demand. Troubleshooting required dashboards, scripts, and manual investigation.

Aparna subsequently entered Cornell’s computer-vision doctoral program, where she explored model bias, then created MonitorML and took it through Y Combinator. Arize acquired MonitorML, bringing her work together with Jason Lopatecki’s efforts to address similar production problems. They founded Arize in 2020.

Her 2021 writing on algorithmic accountability explains why model visibility matters beyond debugging. Excluding protected attributes does not eliminate bias when other variables correlate with them. Aparna argues that model builders need access to appropriate data to test unequal outcomes and must share responsibility for their systems’ consequences. Leaving that responsibility with users asks people to manage risks in systems they cannot inspect.

From model monitoring to agent development

As AI development expanded into language models and agents, Arize developed tracing, evaluation, and experimentation tools for those systems. Arize Phoenix, launched in 2023, provides a self-hosted environment for inspecting traces, assembling datasets, and comparing changes to prompts, models, and retrieval. Arize AX provides managed workflows across development and production. Aparna leads product work and contributes technical writing and experiments; Phoenix and AX are team-built products.

Aparna and Jason jointly announced Arize’s $70 million Series C in February 2025. Dynatrace completed its acquisition of Arize on October 1, 2026, welcoming Aparna as Arize’s chief product officer. The companies plan to connect AI evaluation with observability across the surrounding software and infrastructure, while Arize AX and Dynatrace remain standalone offerings and Phoenix remains available as an open-source project.

Feedback that identifies what to change

Aparna’s evaluation work separates behaviors that an overall score can obscure:

  • Finding information and using it correctly. In research coauthored with Evan Jolley, Aparna examined transformations of retrieved information, including date mapping, currency formatting, and string concatenation. Claude 2.1 initially outperformed GPT-4 on date mapping. Asking GPT-4 to explain its answer before returning it substantially changed the results. Retrieval success alone did not establish generation success, and the model comparison depended on the task and prompt.
  • Choosing a tool and supplying its arguments. Arize’s own copilot selected the correct search tool but passed incorrect parameters. Aparna’s component-level evaluation approach separates those judgments so engineers can locate the mistake. She extends evaluation to tool order, repeated calls, and conversational context: an acceptable final answer can conceal wasted tokens, while repeated questions can reveal that the agent has lost information from earlier turns. Production failures become test cases for comparing revisions.
  • Evaluating the application and evaluating its judge. A judge that misclassifies successful behavior can steer engineers toward the wrong change. Aparna advocates reviewing evaluation outcomes, collecting trusted examples, and refining evaluation prompts alongside application prompts. Explanations help identify misunderstood intent, missing context, or incorrect arguments. Her work also cautions against treating numerical model judgments as a reliably graduated measure of quality.
  • Turning diagnoses into persistent instructions. Her team’s Claude Code and Cline experiments combined generated patches, unit-test results, and explanatory model judgments. A further prompt converted the diagnoses into CLAUDE.md instructions or Cline rules, which the team tested in subsequent runs. Prompt learning changes instructions without changing model weights. The quality of the explanation matters because it determines whether the new rules address a specific failure or merely react to a score.

When agents need adaptive judges

Arize’s assistant Alyx exposed the limits of predefined evaluations as it gained longer memory and the ability to generate interfaces dynamically. It could lose context, fail to recognize completion, or repeatedly call the same tool. Those experiences shaped Aparna’s case for adaptive agent judges. She proposes using them alongside deterministic tests and conventional model judges: fixed checks cover known requirements, while an investigating agent can search changing trajectories for failure patterns the team has not anticipated.

Her June 2026 article with Jason connects that investigation to engineering changes. Signal reviews production traces and groups emerging problems into investigations. Repository-aware managed agents gather context and propose repairs for human review. Full-agent experiments test whether those repairs help without introducing regressions elsewhere; a model upgrade might improve final answers while making tool calls less reliable. Voice agents add audio, interruptions, transcription quality, and speech latency to the behaviors being examined. Engineers review and approve proposed actions.

Aparna and Prukalpa Sankar’s work on context and agent harnesses distinguishes another pair of failure sources. Business definitions and policies determine what an agent knows; its harness governs how it acts through tools, memory, permissions, and handoffs. A capable model can reason coherently from stale information and still produce the wrong result. Evaluating those layers separately helps distinguish a knowledge problem from an execution problem.

Making the traces usable

The feedback process depends on instrumentation that captures enough information to explain a failure. Aparna’s OpenTelemetry analysis, acknowledging contributions from Roger Yang, Xander Song, and John Gilhuly, describes practical mismatches between standard tracing and AI workloads. Evaluation results can arrive after a span has ended, streaming output requires keeping spans open until completion, and attribute limits can discard fields an evaluator needs.

For late evaluation data, she describes augmenting and materializing spans in Arize, while Phoenix joins a separate metadata table to spans. The join approach works for smaller datasets but adds computational overhead at larger volumes. OpenInference extends OpenTelemetry instrumentation to capture AI-specific behavior across frameworks.

In a public post crediting Phoenix’s open-source lead Mikyo King and his small team, Aparna argued that exceptional engineers build systems whose benefits compound. Preserving failures as test cases and diagnoses as reusable instructions gives that position a concrete expression in her work.

Read the topics behind these talks

6 conference talks

Loading more talks and highlights…

References