Sachin Malhotra

Sachin Malhotra builds developer infrastructure and develops practical controls for agents operating production systems. At AI Engineer World’s Fair 2026, he described his work on Anthropic’s CI team: test quarantining, merge automation, CI autoscaling, and merge queues—the machinery engineers use to test, merge, and ship code safely. His approach to agent autonomy replaces broad credential access with operational budgets that limit the volume and pace of actions, account for recoverability, and make mistakes visible.

From Azure DNS to Anthropic’s CI incident responder

Malhotra pursued a master’s degree, during which he worked as a research assistant at the Information Sciences Institute, and held engineering roles at Microsoft and Etsy before Anthropic. He joined Microsoft’s Azure authoritative DNS team in June 2019 and left in May 2022 to join Etsy’s Personalization organization as a staff machine-learning engineer on its Computer Vision team. His reflection on his Microsoft years describes his desire to combine software engineering with machine learning. It also emphasizes work that can be overshadowed by feature development: improving slow builds, neglected integration tests, troubleshooting guides, and missing dashboards. He treats recognizing and repairing those weaknesses as part of becoming a senior engineer.

At Anthropic, Malhotra described his team’s Claude-based CI incident responder, which operates through Slack, investigates failures using engineering tools, and produces situation reports. Repository-managed instructions and a running lessons.md file preserve troubleshooting knowledge between incidents. In a feature-flag incident, the workflow connected diagnosis to a human rollback and then checked that the skip rules had disappeared and the error rate had returned to baseline. The agent’s useful contribution included verifying recovery, rather than stopping at an explanation of the failure. His team also built ci-weather to combine incident updates and CI metrics into shared status reports; making those reports readable still required the team’s judgment about communication.

The cleanup incident behind operational budgets

Malhotra’s World’s Fair talk, “Give the Agent a Budget, Not a Token,” explains why useful operational agents need controls beyond prompts and token scopes. He recounts a cleanup operation in which an empty stage in a command pipeline removed a filter, widening the selection to every workload. The agent deleted about 200 workloads in 90 seconds, affecting resources used by about 20 engineers. It was using his credentials and attempting ordinary cleanup. He locates the failure in giving it unbounded power without close supervision: the agent could perform an authorized operation far faster and more broadly than intended.

Controls for operational autonomy

• Asymmetric permissions: His first distinction is asymmetric permissions: apparently similar actions can have very different failure modes. Re-enabling a quarantined test can turn CI red, producing an obvious signal that engineers can cheaply correct. Skipping a healthy test can let a real bug pass behind green checks. In the quarantining service he describes, agents can re-enable tests, while skipping requires human involvement and an audit trail. He sizes autonomy partly by whether a mistake announces itself or quietly removes a safeguard.
• Replenishing write limits give agents freedom within a bounded allowance. Malhotra advocates a ceiling on every write, with smaller budgets for shared resources than for an agent’s own namespace. Requests above the limit are rejected until the allowance refills. After the deletion incident, a neighboring team—not Malhotra’s own team—built an admission webhook that capped deletions per hour, resource kind, and namespace. Humans retained a bypass, but it refused to operate inside an agent session and directed the agent to ask a human to run the command. Routine work could proceed without approval for each action, while the override remained outside the agent’s control.
• Aggregate tripwires: He pairs those limits with aggregate tripwires. An agent launched separate investigations for dozens of jobs with the same error signature; each investigation looked reasonable in isolation, but together they pointed to a shared infrastructure failure. Monitoring the investigation rate alerted the on-call engineer. Instructions to correlate failures before opening separate investigations helped the agent handle the next occurrence differently. Rate limits constrain activity; tripwires reveal patterns after actions occur and must reach someone who can respond.
• The undo test: His undo test asks whether the agent can reverse an action and whether the impact of a mistake is acceptable. Failure visibility and recoverability are separate questions: a mistake can be obvious yet impossible to undo. In his feature-flag example, an agent can ramp canary traffic from zero to full exposure, inspect reported bugs, and turn the flag back off. It can propose production promotion, but a separately scoped production key remains with a human. An action that cannot be safely reversed requires another party’s authorization and an audit record.
• Infrastructure-assigned identity: These controls depend on infrastructure-assigned identity. Malhotra describes a proxy that holds credentials and stamps outbound calls with an identity the agent cannot choose. Kubernetes jobs receive that identity as a label, and child jobs inherit it, allowing ownership checks, quotas, rate limits, approvals, and monitoring to follow the work. Session identifiers help distinguish multiple agent sessions acting for the same person. If an agent could supply its own identity header, it could claim a new name after reaching a limit and obtain a fresh budget.

Malhotra gives written instructions and infrastructure complementary roles. Prompts explain intent and incorporate lessons from incidents; deterministic controls count operations and reject requests that exceed policy. His objective is to let agents own useful operational loops while keeping the power to exceed consequential limits with a human.