Akshat Bubna
Conference affiliation: Modal
Akshat Bubna is co-founder and chief technology officer of Modal, which he started with Erik Bernhardsson. Modal lets developers run demanding computation across CPUs and GPUs while its team builds the filesystem, container runtime, scheduler and image builder underneath. Akshat’s work explains why that infrastructure matters: an inference service needs capacity before a traffic spike passes, while an agent may need to preserve its workspace for days without paying to keep a computer running.
On this page
From data labeling to cloud infrastructure
Akshat represented India at the International Olympiad in Informatics, earning bronze in 2013 and gold in 2014. He studied computer science and mathematics at MIT and worked at D. E. Shaw and database and fintech startups before joining Scale AI. By June 2021, he was an engineering manager leading a team building software and systems for labeling text, audio and computer-vision data.
His work on human-in-the-loop labeling examined how incentives and human behavior affect the quality of training data. Treating labeling as an optimization problem required accounting for the people doing the work, including the incentives shaping their decisions.
Akshat and Erik began building Modal in 2021 to make it easier to iterate on and ship data, AI and machine-learning applications. Delivering that developer experience led the team into low-level systems work: they built their own storage and execution infrastructure rather than requiring developers to assemble it for each application. Modal became generally available in October 2023, with a Python interface for defining container images, hardware and persistent storage and running workloads such as batch jobs and autoscaling GPU endpoints.
Capacity when it is needed, state when it is not
Akshat’s technical work connects the interface developers use to the behavior of the systems beneath it.
- Fast startup makes elastic GPUs useful. In a technical account co-authored with Charles Frye, Jonathon Belotti and Erik, Akshat explains why autoscaling alone cannot meet variable inference demand. A replica that takes minutes to initialize may become ready after the traffic spike has passed. Modal combines a shared buffer of healthy GPUs, lazy loading of container files, and checkpoint restoration of CPU processes and CUDA contexts to reduce startup work. The goal is to keep paid capacity close to demand while remaining responsive.
- An agent’s workspace should outlive its compute. In his AI Engineer NYC talk, Akshat uses Modal customer Basis’s accounting agent to explain the distinction. The agent may wait days for an accountant’s reply; its documents, workpapers and scripts must survive while its compute stops. Whole-filesystem snapshots, directory snapshots and shared volumes address different needs. Saving a dependency directory, for example, can preserve installed packages while allowing the base image to change.
- Resource allocation should follow the work. Akshat points to CPU and memory measurements from Ramp’s Inspect coding agent that vary both between sandboxes and within a single sandbox over time. Reserving every environment for peak consumption wastes capacity. Modal’s approach lets sandboxes acquire additional resources when needed and pay for that period of use.
Giving agents environments they can manage
Akshat distinguishes the agent harness—the loop that calls models and requests tools—from the environment executing those requests. Separate containers let developers isolate these responsibilities. A sidecar proxy can add credentials to approved requests without exposing the secrets to agent-written code, and restrict particular API paths or operations. Domain allowlists alone cannot express those finer permissions.
He also argues that agents need access to operational information themselves. A training agent should be able to inspect failure logs, check GPU utilization for a stalled run and monitor spending. Those observations help it choose the next experiment or change its resource use.
Akshat’s proposed direction goes beyond scaling identical agent environments. He expects harnesses to create task-specific environments, request GPUs, share files with collaborators and retain snapshots for later work. He cautions against casually generating the sandbox platform beneath them: maintaining isolation, handling rare failures and starting large numbers of environments require sustained systems engineering. His bet is that well-built compute, storage and networking primitives give agents more room to adapt their own work.
1 conference talk
Loading more talks and highlights…
