AI Engineer World's Fair 2026

Agents at Scale: Inside MiniMax's Model and the Infrastructure Behind It

Loading slides…

Read the talk

Agents at Scale: Inside MiniMax's Model and the Infrastructure Behind It

MiniMax’s Olive Song and Together AI’s Dan Fu connect multimodal training, useful kernel benchmarks, long-running agent evaluation and the infrastructure needed to serve them.

From a talk by Dan Fu and Olive Song

Before you start: Familiarity with language-model inference, reinforcement learning and GPU kernels will help; no knowledge of MiniMax’s architecture is required.

What happens between training a model and serving it?

A new model arrives. What has to happen before developers can use it reliably—and what keeps changing after launch? Dan Fu and Olive Song work on opposite sides of that handoff. Dan, Together AI’s VP of Kernels, leads inference and GPU optimization. Olive, MiniMax’s RL research lead, is responsible for final training and shipping: the work immediately before inference.

The moderator introduces MiniMax-M3 as MiniMax’s strongest model yet and asks why the company releases its weights. Olive’s answer connects access with improvement. Developers can use the model, contribute feedback and pull requests, and help strengthen it. Inference specialists can optimize the same weights to make them faster to serve. Open weights make the training–serving handoff a place where other teams can contribute, rather than leaving every improvement to the model’s creator.

Two panelists beside a World's Fair lectern against blue curtains; the panelist on the right gestures toward the audience.Open full source frame
Panelists onstage at AI Engineer World's Fair.
0:330:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:33 · section reference included

From an informal introduction to architecture work

The moderator describes the distribution effort as having the lion’s share of M3 token usage that morning, without supplying a measured share or denominator. Dan frames Together’s objective more broadly: make more tokens available so more people can do useful work. The partnership itself began informally. At a car event in Las Vegas the previous year, a MiniMax representative encouraged Together to serve the company’s next model.

Together subsequently served MiniMax-M2.5 and MiniMax-M2.7. By the approach to M3’s launch, the team had seen enough usage and developer activity to be excited about the next release. Collaboration then became concrete engineering work: studying the model’s architecture, optimizing it and determining how best to serve inference. The relationship was not simply a distribution agreement; it gave the serving team a path into the model’s technical decisions.

2:262:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:26 · section reference included

Capabilities depend on how the task is defined

M3’s major change from the M2 series is multimodality. Alongside understanding text and writing code, it understands images and videos. Olive highlights two applications:

  • Computer use: navigating a computer and using its tools to create things.
  • Game development: a capability MiniMax deliberately worked on, and one Olive considers an underappreciated use of the model.

These examples broaden the task beyond producing a text answer. The model must work through tools and, for visual applications, understand what those tools produce.

The moderator follows with SVG-Bench, KernelBench and OSWorld, asking what domain-specific post-training requires. Olive starts with data and problem definition, not a single training recipe. For kernel optimization, the environment must let the model work on kernels and iteratively improve their performance. Designing that environment is part of defining the learning problem: it determines which actions the model can take and which improvements reinforcement learning can discover.

4:144:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:14 · section reference included

A benchmark whose solutions can become infrastructure

Dan reports that both open and closed models are getting better at writing kernels, and that Together already uses models when developing kernels and optimization frameworks. ParallelKernelBench takes the next step: it includes problems Dan describes as unsolved. The team surveyed ways of serving model inference and found potential speedups for which good kernels did not yet exist. Those gaps became benchmark targets.

That changes the incentive behind optimizing for a benchmark. Normally, overfitting raises the concern that a score improves without producing something useful outside the test. Here, Dan welcomes solutions tailored to the benchmark because Together can take the resulting kernels and use them to accelerate inference and development. The benchmark is valuable when solving its tasks produces usable engineering artifacts. The moderator extends the idea: if benchmarks cover enough practically useful problems, bringing those problems into the model’s training distribution can itself be desirable.

6:016:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:01 · section reference included

Day zero starts an optimization backlog

Serving work begins before launch, when a partner shares architecture details. For M3, those details included MiniMax Sparse Attention. Attention is only one source of variation: open models also differ in mixture-of-experts choices and quantization. A serving engine cannot assume that every new model is the previous one with different weights.

Dan describes a practical sequence:

  1. Inspect the architecture. Identify the operations the model needs.
  2. Benchmark available kernels. Determine which existing implementations work, which need modification and which must be written from scratch.
  3. Protect launch quality. Check that the served model delivers the expected quality and user experience.
  4. Work through the backlog. Continue improving the KV cache, attention kernels and quantization after release.

Dan describes successive improvements over the first weeks after launch, but gives no numerical speedup. The launch is a working starting point, not the end of performance engineering.

The moderator illustrates the pace with a conversation with Ingrid. Asked whether M3’s performance had improved—meaning over the previous month—she interpreted the question as asking about progress since the previous night. The anecdote captures how short the optimization cycle can become once a model is serving real traffic.

7:227:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:22 · section reference included

Agents change what the inference engine must optimize

The workload is changing at the same time as the models. Instead of a person sending occasional chat turns, a harness can drive hundreds of tool calls across a continuing task. Dan says this affects KV-cache design, prompting and the choice of which parts of the stack to optimize. His contrast is between a relatively modest system prompt plus chat history and a coding workflow that supplies an entire codebase.

WorkloadContext patternServing concerns
ChatSystem prompt and conversation historyServing successive chat turns
Coding agentCodebase and repeated tool interactionsKV cache, routing and kernel behavior

The important difference is the shape of the work, not just a longer prompt. The engine must serve repeated interactions around substantial context. Together follows those workload changes to decide how its inference stack should evolve.

9:049:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:04 · section reference included

Train the model to see what its code produces

The moderator offers a concrete multimodal agent loop: ask a coding agent to improve a web application, have it use the application, then feed the observations back into another iteration. A model that only reads source code misses part of the evidence. To improve the rendered interface, it helps to see and understand the interface itself.

Olive explains that M3 trained on text and images together from the first step. She says MiniMax overcame training collapse of a kind encountered at other labs, though she does not describe the remedy. The observable behavior she points to is in the attention maps: text tokens attend to visual tokens, allowing the two forms of information to interact.

Website development makes that interaction useful. The model can inspect how a website looks and use that information to improve it, including during reinforcement learning. In the moderator’s loop, the rendered application becomes evidence for the next coding decision. Training text and vision together supports the connection between an implementation and its visible result.

10:1410:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:14 · section reference included

Architecture changes, but engineering experience transfers

The moderator cites a one-million-token context window alongside multimodality and sparse attention as simultaneous changes in M3. That combination creates many possible optimization targets. Asked where to focus among a thousand possibilities, Dan jokes about working on a thousand and one. His answer emphasizes searching for every available improvement; it does not identify a ranked set of bottlenecks or a single dominant optimization.

That breadth does not mean starting over for every model. Dan says MiniMax Sparse Attention differs from the variants found in DeepSeek and GLM, but lessons from optimizing those variants still transfer. What carries forward is experience with the optimization process and kernel writing, even when the specific operation changes.

Dan connects that accumulated experience to work extending back to his PhD. Seeing these ideas trained at scale and used in deployed models is a form of validation. The moderator places it within a wider pattern: open-model labs can learn from one another’s successful choices, letting useful architectural and engineering ideas spread.

11:4411:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:44 · section reference included

Long research tasks need intermediate evidence

The moderator cites MiniMax’s launch account of a 12-hour run reproducing an ICLR paper. The discussion does not specify the paper or its acceptance criteria, so this is a reported example rather than a reproduction success rate. It raises a harder training problem than a short coding task: paper replication and kernel-framework optimization can require long sequences of decisions, GPU access and constrained hardware resources.

The training answer returns to the same foundations as kernel post-training: formulate the problem, choose the data, define the rewards and build the environment. Long tasks also motivate changes to the RL algorithm to make training more efficient. The discussion describes capabilities emerging across RL iterations, without giving a complete algorithmic recipe.

Evaluation need not wait until the entire run ends. The model can submit several times, and each submission can be evaluated. Validation and test splits help distinguish actual improvements from attempts to exploit the evaluation. Intermediate submissions provide evidence of progress; held-out checks test whether that progress survives beyond the target the model has been optimizing.

A small Python evaluator can preserve that separation in its result records:

python

from collections.abc import Callable, Iterable
from typing import TypeVar

Candidate = TypeVar("Candidate")


def evaluate_submissions(
    submissions: Iterable[Candidate],
    validation_score: Callable[[Candidate], float],
    test_score: Callable[[Candidate], float],
) -> list[dict[str, int | float]]:
    records = []
    for submission_id, candidate in enumerate(submissions):
        records.append({
            "submission_id": submission_id,
            "validation_score": validation_score(candidate),
            "test_score": test_score(candidate),
        })
    return records

This expresses the evaluation bookkeeping, not the training algorithm. The harness still determines the task-specific scoring functions and which results become model feedback; keeping a separate test score is useful only if the test remains independent of the optimization process.

The discussion then connects evaluation to M2.7’s self-evolution work. Using models to accelerate internal development creates another source of tasks: the researchers’ own work. Those tasks can become internal evaluations, making model assessment closely related to the development activities the team actually wants to improve.

13:5914:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:59 · section reference included

The KV cache becomes a distributed storage problem

Long-running agents return the conversation to serving infrastructure. With concurrent requests and very long contexts, the KV cache keeps growing. Dan compares managing it to building a distributed file system or a very large database. The question is no longer only how fast a kernel runs; it is also how the system stores, finds and moves the state that inference needs.

Dan reduces the problem to familiar storage questions:

  • Placement: Where should the cache live?
  • Recognition: Has the system seen this content before?
  • Retrieval: How does it fetch the needed cache?
  • Movement: How does it transfer cache between locations?

He jokes that the industry is rediscovering material from an undergraduate systems class. The concepts may be familiar, but their implementation still has to be careful: a good cache design must connect those storage decisions to the inference work that consumes the cached state.

16:1116:24
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:11 · section reference included

Better hardware use and faster model development

Referring to Dan’s Stanford lecture, the moderator asks what today’s practices might look like three years later. Dan’s first answer is GPU underutilization. He loosely cites roughly 10% FLOP utilization attributed to SpaceX, without identifying the workload or measurement conditions. His broader expectation is that both training and inference should use available hardware much more effectively; progress need not depend entirely on getting new GPUs.

Dan also hopes the recurring debate about whether open models can catch the closed frontier will be settled. He points to M3, GLM and Kimi as reasons for optimism about competing with Anthropic and OpenAI. This is his assessment of the direction of progress, rather than a benchmark comparison presented in the discussion.

The moderator turns the same future-looking question toward post-training, describing self-evolution as a model building a harness, training inside it and producing a next checkpoint. That shorthand is best understood as model-assisted development, not proof of an entirely autonomous training pipeline. Olive responds by recalling that three years earlier she was still in school and had not entered the industry; the present pace would have been difficult to imagine.

What she can already observe is models accelerating development, a process she says was underway a year or more earlier. As those tools improve, the work of developing subsequent models can move faster too. Olive connects that increasing speed to open-weight models catching frontier labs—and to MiniMax’s original purpose for releasing them: making the resulting capabilities available for everyone to use.

17:1617:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:16 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…