← All speakers

Simran Arora

Conference affiliation: Together AI · 2026

Simran Arora researches efficient AI models and the systems that run them. Her collaborative work connects what a language model can remember with the memory, computation, and communication its implementation requires. Projects including Based, ThunderKittens, Cartridges, and ParallelKernelBench examine these costs alongside the capabilities an application needs.

On this page

Training and early work on personalization

Arora earned her undergraduate degree at the University of Pennsylvania in 2019 and her computer-science doctorate at Stanford in 2025, working with Christopher Ré in the Hazy Research Lab. She also co-created and taught Stanford’s Systems for Machine Learning course in fall 2023. At her 2026 AI Engineer talk, she described herself as a principal scientist leading Together AI’s Frontier Performance research team and an incoming professor at Caltech.

Her earlier research explored how foundation models could support private, personalized applications. In writing with Christopher Ré on privacy and personalization, she examined approaches to using personal information without requiring large private training datasets or sacrificing privacy guarantees. Collaborative research on retrieval across public and private information addressed the different data distributions those systems must handle. Knowledge access, application quality, and cost were connected design problems.

Recall, efficient architectures, and reusable context

That concern with access to information also shaped her work on efficient language models. Arora, Sabri Eyuboglu, Michael Zhang, and their collaborators developed Zoology to investigate why some alternatives to Transformers struggled to recall information from their prompts. Broad metrics could conceal failures on predictions requiring an earlier association. Simple synthetic tests initially missed the weakness; tests requiring multiple lookups exposed how the resources needed for recall changed with sequence length. A cheaper layer could require a wider model to recover the missing capability, making task-level resource requirements more informative than per-layer complexity alone.

  • Based: recurrent memory: These findings helped motivate Based, a recurrent language-model architecture combining local mixing with linear attention. Local operations handle nearby relationships, while linear attention carries information over longer distances through a fixed-size recurrent state. The architecture gives researchers ways to trade memory and throughput against recall. In her one-year retrospective, Arora emphasized careful characterization of those tradeoffs: broad claims about efficient architectures can obscure the tasks on which they still struggle.
  • Cartridges: reusable context: With Cartridges, Arora and her collaborators approached repeated context as reusable memory. They train a compact key-value cache offline for a corpus such as a codebase or document collection. Their self-study method generates synthetic conversations about the material and distills context-conditioned behavior into the cache. Multiple requests can then reuse the cartridge, spreading preparation costs across queries. Simply training on next-token prediction from the corpus proved insufficient; the training method mattered to whether the stored context was useful.

From GPU kernels to multi-GPU communication

Arora’s GPU programming work addresses whether such efficiency gains translate into fast execution. She co-authored ThunderKittens, which organizes kernel programming around matrix tiles and templates for overlapping work. Its abstractions follow the GPU execution hierarchy, from operations within a warp to coordination across thread blocks, making kernels easier to express while retaining control over hardware resources. She also co-authored HipKittens, which develops tile-based abstractions for high-performance AMD GPU kernels.

As single-GPU execution improved, Simran saw communication between GPUs become a larger bottleneck in distributed training and inference. Libraries tuned for bulk transfers can leave room for custom kernels that overlap smaller transfers with computation. She and her collaborators studied those tradeoffs by hand, then built ParallelKittens around a small set of patterns that let programmers choose how data moves and how to schedule transfers alongside computation.

Simran and her collaborators built ParallelKernelBench to test whether language models could turn distributed PyTorch references into fast CUDA kernels. Across 87 problems drawn from real AI workloads, the benchmark checks correctness separately from speed relative to the reference. In the results Simran presented, extra attempts and an agent harness yielded more correct kernels, but performance gains leveled off. Models had an easier time fixing compilation errors than deciding how to partition data, order communication, and overlap transfers with computation. Successful kernels tended to follow familiar patterns.

The benchmark’s practical aim is to produce useful kernels for workloads that have received less manual optimization. Arora highlighted generated examples involving NeMo vocabulary-parallel filtering, Hyena context parallelism, and intersection-over-union suppression in SAM 3 video segmentation. Her work makes the remaining challenge concrete: execution feedback can help a model produce a working program, while fast multi-GPU execution still requires choosing communication and scheduling strategies that fit the hardware and workload.

Read the topics behind these talks

1 conference talk

Loading more talks and highlights…

References