← All speakers

Milind Sunil Shah

Conference affiliation: Brainbase Labs

Milind Sunil Shah is a San Francisco–based software engineer, founder and founding member of the technical staff at Brainbase Labs. He leads engineering for infrastructure powering more than 10,000 internal AI agents, as described in his 2026 AI Engineer NYC biography. His work with a regional Domino’s pricing team shows what that engineering involves: Milind and his colleagues improved an agent from 41% to 97% benchmark accuracy across eleven versions by changing instructions, skills and playbooks while keeping the same model.

On this page

From software integrations to AI development infrastructure

As an implementation lead at Qure.ai from October 2020 to May 2021, Milind built backend services and integrations for hospital and radiology systems. He worked with clients to understand their workflows, wrote APIs for their information systems and used DICOM, the standard for exchanging medical images. His other engineering work includes cloud governance tools at Adobe and developer infrastructure at DoorDash. He holds a master’s degree in computer science from the University of Massachusetts Amherst.

At Mercor, Milind built post-training data pipelines and environments for AI coding agents. He described fragmented data systems and inadequate tooling as obstacles to the research work those systems were meant to support. He subsequently joined Pipelines as co-founder and CTO, alongside CEO and co-founder Piper Cyterski, to build infrastructure for data collection, evaluation and reinforcement-learning training environments. His stated aim was to help teams spend more time answering research questions and less time assembling the tools needed to investigate them.

At Brainbase, Milind works on infrastructure for operating agents inside businesses. The platform connects agents to tools, runs scheduled or event-triggered tasks, and supports evaluations and human review. His Domino’s work explains how his team applies those capabilities to a task where a completed spreadsheet can still contain wrong answers.

Defining what a correct answer means

In his account of the Domino’s project, Milind starts with the analyst who must approve the result. The regional pricing team wanted the lowest price a customer could pay for competitors’ products across websites and delivery apps. Before automating that search, the engineers needed to establish what belonged in each cell, which assumptions the agent could make and what it should flag for review.

Delivery and takeaway were separate targets. A crossed-out list price was different from the amount paid at checkout, and a percentage discount required arithmetic before the result entered the spreadsheet. The team stored observations against item IDs, with the source and selection rule attached, so an analyst could trace an incorrect cell back to the decision that produced it. If the agent could not verify a price, it was instructed to record a gap rather than guess.

Milind and his colleagues began with one item on one app in one city, measuring accuracy, cost and runtime before expanding. Three decisions shaped the subsequent work:

  • Preserve cases that can be checked again. The team tried reconstructing historical tasks from analysts’ spreadsheets and archived websites. Delivery apps did not expose the necessary historical information that way, so the team began retaining browser screenshots from agent runs to build its benchmark environment.
  • Match the check to the mistake. Code checked spreadsheet completeness, discount arithmetic and whether large price changes were flagged. An LLM examined traces for item selection and promotion use. A reviewing agent revisited flagged prices, with analysts reviewing the remaining exceptions.
  • Change behavior in response to observed failures. In one run, the agent stopped at the section linked on a pizza page and missed a cheaper $21 deal farther down. The team added a rule to inspect the whole page. On another site, the agent repeatedly enlarged promotional images even after finding the right card. Reviewing the trace led the engineers to require checking the page’s HTML structure, image titles and alternative text before reading images, and to limit further zooming. Milind reported that this reduced image reads to one-third and cut the cost of that task by more than half.

The handoff included a way to keep improving the system. Analysts eventually reviewed exceptions and could turn corrections into benchmark cases. For Milind, deciding who reviews outputs, converts mistakes into rules and approves new versions belongs in the deployment work itself.

Making group decisions explicit

Milind also built Booth, a web app for friends choosing where to eat or go out. It addresses a familiar problem with group chats: venue suggestions disappear among messages, votes scatter and someone has to keep track of dietary requirements.

Booth gives participants a shared place to suggest venues, approve multiple acceptable options and explain vetoes. Dietary preferences stay with each participant’s profile, while an AI participant named Nomi can suggest places and check them against the group’s requirements. A veto requires a named, written reason, giving the group an objection it can address. The project makes a specific design choice visible: conversation, preferences and the decision need a shared structure if people are to act on them.

1 conference talk

Loading more talks and highlights…

References