← All speakers

Nishant Gupta is a distributed-systems engineer focused on making autonomous AI agents reliable, observable, and safe in production. In 2026, he was a software engineering technical lead at Meta, working on training and inference infrastructure associated with Meta Superintelligence Labs.

Gupta earned a master’s degree in computer science from the University of California, Los Angeles, in 2019. In 2024, he was the first-listed author of research on safely oversubscribing Meta’s datacenter capacity. The work explored dynamic idle resource leasing: assigning unused computing capacity to additional workloads while protecting reliability and service-level guarantees. That background in elastic scheduling and resource allocation informs his approach to agents whose compute requirements shift with reasoning, tool use, and workload complexity.

  • Deterministic execution boundaries. Models should propose actions while independent infrastructure validates requests, enforces policies, and controls access to production systems. Gupta connects agent reliability architecture to established distributed-systems safeguards, including controlled retries, circuit breakers, resource quotas, and isolated execution.
  • Agentic control planes. Scheduling, workload routing, shared memory, policy enforcement, observability, and evaluation need a coordinated operational layer. Gupta identifies runaway retry loops, escalating compute costs, stale reads, and conflicting updates as infrastructure failures that can masquerade as poor model reasoning.
  • Scenario-driven production evaluation. Agent performance depends on complete workflows: planning, tool execution, failure recovery, task completion, latency, cost, safety, and escalation. His approach to evaluating production agent behavior uses realistic scenarios, distributed traces, and live operational signals to expose failures that isolated answer-quality benchmarks miss.
  • Targeted human oversight. Human reviewers resolve ambiguous failures, assess consequential decisions, and calibrate automated evaluations. Continuous telemetry and feedback help direct their attention toward situations where autonomous systems cannot reliably govern themselves.

Read the topics behind these talks

3 conference talks

Key ideas

Scroll to read ↓

An agent that can complete a workflow still needs infrastructure that bounds its failures, controls its actions, coordinates its state, and makes recovery possible.

  • Can the workflow run reliably?
    0:03 ↗
  • How an invalid tool call becomes a compute incident
    2:06 ↗
  • Give the model proposals, not production authority
    3:08 ↗
  • A control plane needs a decision history
    3:30 ↗
  • Coordinate memory, layer safety, and route exceptions
    4:23 ↗
  • Inference becomes a scheduling problem
    5:36 ↗
  • Adapt the protections distributed systems already use
    6:03 ↗
  • The differentiator moves into infrastructure
    6:27 ↗

Key ideas

Scroll to read ↓

Evaluating an agent means checking whether its workflow delivers dependable outcomes, from tool execution and failure recovery to the cost of keeping it running.

  • From benchmark scores to system behavior
    0:22 ↗
  • Failure layers and the SRE lens
    2:10 ↗
  • Benchmarks, scenarios and production evidence
    3:14 ↗
  • Production signals need human judgment
    4:14 ↗
  • Detecting drift requires workflow traces
    5:06 ↗
  • The evaluation loop continues after deployment
    6:10 ↗
  • Choose metrics by the outcomes they represent
    6:38 ↗
  • Evaluation becomes infrastructure
    7:12 ↗

Key ideas

Scroll to read ↓

Nishant Gupta and Naman Ahuja explain how routing, caching, scheduling and reliability interact in distributed inference—and why efficiency should be measured by cost per successful task.

  • Agent capacity planning must account for calls per user and tokens per call, along with hardware and model choices. User count alone misses the demand multiplier.
    2:06 ↗
  • Routing changes can propagate through cache hits, batch composition and GPU utilization into autoscaling decisions. Diagnose regressions across the stack.
    5:50 ↗
  • Workflow-aware scheduling considers hardware readiness, request state, tenant objectives and completed work. Losing step three can waste the spending on steps one and two.
    8:31 ↗
  • Retry budgets, routing circuit breakers, admission control and queue-based load shedding interrupt cascades. Cold capacity needs time to become useful.
    12:36 ↗
  • Use telemetry to coordinate serving decisions and optimize cost per successful task. Lower token cost can lose its value when failures, retries or poor responses prevent completion.
    11:47 ↗

References