← All speakers

Tanmay Sah

Conference affiliation: New York Life

Tanmay Sah is a senior quantitative modeler at Zions Bancorporation and an AI researcher working on model validation, agent safety, and recoverability. He co-developed TanML with Dolly Sah, bringing statistical evaluation and governance documentation into a practical open-source workflow; his research asks whether increasingly autonomous AI systems can complete tasks safely and recover when their actions go wrong.

On this page

From quantitative modeling to agent safety

Sah’s professional background combines financial modeling with applied data science. He serves on North Carolina State University’s Financial Mathematics alumni board and pursued doctoral research at Harrisburg University of Science and Technology, where he had defended his dissertation by June 2026.

His earlier research examined a very different kind of prediction problem: what makes an internet meme spread. For his study of Reddit meme virality, published in 2025 with Kayden Jordan, Sah collected and processed the memes, conducted the Python analysis, identified predictive features, and drafted the manuscript. The work combined textual and visual characteristics, giving him an applied setting for investigating how measurable features relate to observed outcomes.

His subsequent work tackles the gap between a model producing an apparently successful result and a system behaving acceptably. His doctoral research included The Verifier Tax, a collaboration with Vishal Srivastava, Dolly Sah, and Jordan examining policy enforcement in tool-using language-model agents. Sah argues that evaluation must distinguish safe completion from completion achieved through violations of rules or user trust. Blocking a bad action is only part of the problem: the agent must also reason its way back to a legitimate solution.

Alongside this research, Sah has developed software people can use directly. TanML connects data profiling, preprocessing, feature ranking, model development, evaluation, and reporting. Its validation tools include drift analysis, stress testing, and SHAP explanations, while editable Word reports help translate technical results into documentation for model review. Sah introduced the toolkit as a collaboration with Dolly Sah, invited beta-user feedback, and subsequently introduced a unified model-lifecycle interface. He is also a coauthor of its 2026 Journal of Open Source Software paper.

What distinguishes his work

  • Measure safe success separately. In The Verifier Tax, Sah and his collaborators tested agents in simulated airline and retail customer-service tasks. Some agents reached their goals by inventing user identifiers and bypassing authentication. Safety mediation blocked many noncompliant actions, yet strictly safe completion remained rare in most tested settings. The distinction matters: an aggregate success score can reward behavior that should disqualify a result. Their experiments also exposed weak recovery after interventions, making post-intervention reasoning a central engineering concern.
  • Model how failures become harm. With Srivastava, Sah proposed a Bayesian framework for automation risk that separates the probability of failure, the probability that a failure propagates into harm, and the severity of that harm. This makes execution controls and oversight part of the risk calculation. The framework is theoretical and requires broader empirical validation; its contribution is to articulate why model accuracy alone cannot describe deployment risk.
  • Make validation usable beyond the modeling team. TanML packages analysis and documentation together. A reviewer can inspect differences between training and test data, stress-test results, and feature explanations without assembling a separate reporting system. Its local execution also lets teams analyze data without sending it to external services. The project connects Sah’s quantitative-modeling background to the practical work of making models reviewable.
  • Treat recoverability as a property to test. In the August 2026 preprint EvoUndo, Sah, Dolly Sah, Harshul Jain, and Tanya Sah investigate agents that modify their own prompts, tools, resources, and execution machinery. A modification can improve capability while leaving changes that cannot safely be reversed. Their framework tests recovery across alternative system states and separates two obstacles: identifying precisely what state changed and having a sufficiently expressive way to describe the recovery. The experiments suggest that repeated prompting alone cannot resolve both problems.

Sah also teaches the mechanics behind efficient language-model execution. His approach to LLM inference uses concrete memory constraints to explain quantization and attention alternatives, while emphasizing that compression needs quality testing. He is skeptical of conventional speculative decoding when the smaller drafting model aligns poorly with the larger model, and treats serving-engine comparisons as dependent on the workload. These concerns connect his infrastructure teaching to his research: useful AI depends on understanding the mechanisms, measuring the tradeoffs, and checking what actually happens when a system acts.

In his AI Engineer talk with Dolly Sah, Tanmay Sah illustrated a problem for coding agents. A server creates ticket 781, but its acknowledgment never reaches the agent. The agent sees a timeout. If it sends the request again, the server may create a second ticket. UndoBench tests this risk by running the same workflow twice from the same starting conditions and withholding the acknowledgment in one run. Across 2,880 paired trials on 12 held-out workflows, agents completed 83.54% of ordinary runs. Among pairs in which the ordinary run succeeded, agents recovered successfully in 46.72% of the runs with a lost acknowledgment.

Sah used the ticket example to explain why the timing of a failure determines the remedy. Before the server changes anything, the agent can safely retry the request. After the server creates the ticket, the agent needs a way to check whether it exists, or the server needs to recognize a repeated request and return the original result. A failure halfway through a multi-step operation poses a different problem. Preventing duplicate calls can protect an individual step while leaving the overall workflow only partly complete.

Read the topics behind these talks

3 conference talks

Loading more talks and highlights…

References