← All speakers

Bio, Work & Ideas

Shreya Rajpal

Conference affiliation: CEO · Snowglobe · 2026

Shreya Rajpal is the co-founder and chief executive of Guardrails AI, creator of the open-source Guardrails framework and Snowglobe, a simulation platform for testing AI agents. She builds infrastructure that helps companies detect unreliable model outputs and agent failures before they reach customers.

Rajpal studied artificial intelligence at the University of Illinois Urbana-Champaign and worked on self-driving technology at Drive.ai and Apple’s Special Projects Group. She later became a founding engineer at Predibase, where she led machine-learning infrastructure. Autonomous vehicles helped shape her conviction that consequential AI systems need explicit verification and simulated testing before deployment.

In March 2023, she introduced Guardrails, an open-source package for validating language-model outputs, enforcing structured responses, and requesting corrections when results violate application requirements. The Guardrails framework turns verification-first AI development into a practical architecture: inspect each response, then accept, repair, retry, suppress, or log it.

Her technical position is unusually concrete: prompts and fine-tuning can improve behavior, but neither guarantees that an individual answer is accurate, safe, or correctly structured. Validators should match the actual risk, whether that means screening private information, executing generated code, testing SQL against a sandboxed database, or checking customer-support answers against trusted documentation. Rajpal also teaches these principles through a DeepLearning.AI course on reliable AI applications.

In August 2025, she introduced Snowglobe, extending verification from individual model responses to complete agent interactions. The simulation platform generates synthetic users and multi-turn conversations around real agents, allowing teams to test behavior without experimenting on customers.

  • External grounding makes verification meaningful. Generated answers should be checked against authoritative documents, executable environments, schemas, or other application-specific evidence—not judged solely on whether they sound plausible.
  • Synthetic agent evaluation requires coherent state. Useful simulations combine believable personas, mocked tools, consistent account information, and interactions whose earlier decisions affect subsequent turns.
  • The simulation-to-production gap must be measured. Synthetic results deserve trust only when calibrated against human judgment and real-world outcomes. In work with Nubank engineer Aman Gupta, Rajpal described Snowglobe’s infrastructure; reported customer-service improvements and deployment acceleration belonged to Nubank.
  • Strong evaluation enables high-throughput experimentation. Rajpal identifies evaluation design and rapid experimentation as essential AI-engineering skills: aligned metrics and readily generated test data let teams compare changes, isolate failures, and improve agents before release.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

Nubank uses simulated customers and consistent mocked tool state to test agent changes before live experiments, shortening the wait for useful evaluation signal.

  • Improving support without waiting on every live experiment
    0:32 ↗
  • An agent example is a trajectory
    2:21 ↗
  • Two ways to obtain data, two different costs
    4:49 ↗
  • Remove the wait for feedback
    5:46 ↗
  • Run the real agent inside a simulated environment
    7:20 ↗
  • Maria Souza: a persona with consistent account state
    8:42 ↗
  • Evaluate real and simulated conversations together
    10:10 ↗
  • Check whether simulation tracks production
    11:14 ↗
  • Screen candidates while preserving self-service
    11:55 ↗
  • Shortlist models before testing them live
    13:33 ↗
  • Continuous improvement depends on trustworthy signal
    14:26 ↗

Key ideas

Scroll to read ↓

A prototype can look reliable until someone else uses it. Application-specific checks and corrective retries give model outputs a clearer path into dependable software.

  • The prototype works—until someone else tries it
    0:15 ↗
  • Variable outputs become compounding errors
    2:50 ↗
  • Insert verification before consumption
    5:50 ↗
  • Separate the check from the failure policy
    9:10 ↗
  • Context is not a compliance check
    11:43 ↗
  • Training and model updates add operational work
    12:48 ↗
  • Choose a checker that fits the constraint
    13:49 ↗
  • Define correctness for a help-center chatbot
    15:48 ↗
  • Repair an unsupported password-change answer
    17:46 ↗
  • Make application policy executable
    18:44 ↗

References