AI Engineer World's Fair 2026

Data and Environment Curation for Post-training LLMs

Loading slides…

Read the talk

Data and Environment Curation for Post-training LLMs

A wrong tool call can interrupt an otherwise capable agent. Better post-training starts with testing which questions, reasoning traces and environments actually improve the model.

From a talk by Mahesh Sathiamoorthy

Before you start: Familiarity with language-model prompting and fine-tuning will help; SFT, rollouts and the role of RL environments are explained as they appear.

Connect data curation to model improvement

How do you know whether the data you curate will actually improve a model? Mahesh Sathiamoorthy introduces this problem through Bespoke Labs, where he identifies himself as co-founder and CEO after working as a researcher and engineer at Google DeepMind. The team’s work progressed from Curator, a tool for synthetic supervised fine-tuning data, to Bespoke Stratos after DeepSeek arrived, and then to OpenThoughts. Contributions to Terminal-Bench, RL environments and enterprise custom models extended that work from reasoning answers to agent behavior.

“Who & What” slide describing Bespoke as an applied data research lab and listing data curation, OpenThoughts, Terminal Bench, RL environments and enterprise post-training.Open full source frame
Bespoke’s research focus and projects, including Curator, OpenThoughts and Terminal Bench.

Data curation needs feedback from training. Data producers and researchers often occupy separate roles, but a dataset that looks useful to its creator may not move the model’s metrics. Sathiamoorthy advocates doing both: curate examples, train on them, and use the resulting measurements to decide what to curate next. That connection is the organizing principle behind the recipes that follow.

0:210:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:21 · section reference included

From knowing answers to completing work

The shift from knowledge benchmarks to agent benchmarks changes what training data must support. The slide contrasts MMLU’s tests of knowledge across subjects with SWE-bench’s tests of software work; Terminal-Bench likewise asks what an agent can accomplish. Answering a question correctly is only part of the requirement when a model must act autonomously.

MMLU paper header above a SWE-bench paper header, connected by a downward arrow, with labels “What do they know” and “What can they do?”Open full source frame
From what models know to what they can do: MMLU and SWE-bench.

Longer autonomy exposes the next constraint: reliability. An agent may call the wrong tool or make another mistake that interrupts its work. Hours, days and weeks of autonomy are goals here, not reported achievements. Reaching those horizons requires reducing the failures that accumulate during execution.

There are several places to intervene:

  • Prompts: improve the instructions guiding the agent.
  • Harness and tools: change the execution machinery and available actions.
  • Model training: use post-training to improve the behavior the model brings to the task; stronger pretraining can also contribute.

Reinforcement learning is one popular post-training method, which makes the environments in which agents practice an essential part of the reliability problem.

2:402:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:40 · section reference included

Environments are another form of training data

Sathiamoorthy uses data broadly enough to include RL environments. Supervised fine-tuning, or SFT, consumes demonstrations; RL environments provide settings in which a model can generate experience. The training input has changed shape, but someone still has to construct it and establish its quality.

His diagnosis is that high-quality data and environments are often the bottleneck, especially for enterprises. Compute, capable starting models and training infrastructure are comparatively established, with providers such as Fireworks and Tinker supplying parts of the infrastructure. Frontier labs face the same demand for useful environments even when their training systems are already in place.

The payoff is broader than benchmark capability. Post-training can also target latency, cost and throughput. The examples move through reasoning-data curation, agent trajectories and environments, an enterprise deployment, and finally the tooling needed to run this process.

4:585:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:58 · section reference included

Build a reasoning-data recipe through ablations

OpenThoughts began with a gap: after DeepSeek’s arrival, the open community lacked comparable access to high-quality reasoning data. Bespoke Stratos expanded into a collaboration with researchers at Stanford, UC Berkeley, UW and other institutions. The resulting OpenThoughts paper, published at ICLR 2026, documents a recipe for constructing that data.

Sathiamoorthy reports that increasing dataset size under the OpenThoughts recipe improved evaluated performance, including on AIME and LiveCodeBench. This is the relevant scaling test: does producing more data with the same recipe continue to help? He also cites Microsoft attention and John Schulman’s report of internal use at Thinking Machines as signs of adoption, separate from the experimental evidence.

The pipeline contains several decisions that must be tested rather than assumed:

  1. Collect source questions. Start with existing prompt-response datasets and extract their prompts.
  2. Choose the mixture. For a target such as 10,000 samples, decide how many questions to draw from each source.
  3. Select questions. Test LLM judgments of quality and difficulty, then mix and filter the questions.
  4. Generate reasoning answers. Use teacher models, such as DeepSeek, Qwen-based models or Gemini, to produce responses.
  5. Test answer selection and multiplicity. Compare filtering answers with retaining them, and generating one answer per question with generating several.
  6. Run staged ablations. Measure the effect of choices at each stage and carry useful choices forward into the next experiment.

The final recipe is the result of these comparisons, not simply the pipeline diagram.

6:517:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:51 · section reference included

More answers can matter as much as more questions

One productive choice was to sample multiple answers to the same question. The comparison is between spending a generation budget on more distinct questions answered once and answering a smaller set of questions repeatedly—for example, sixteen times each. Sixteen is a tested choice, not a universally optimal setting.

A small Python helper makes that allocation explicit. Here, 100 questions answered sixteen times and 1,600 questions answered once each create 1,600 generation requests. The sample_index distinguishes requests for the same prompt; the teacher’s generation settings determine whether the resulting traces actually differ.

python

from collections.abc import Iterable


def generation_requests(
    questions: Iterable[str], samples_per_question: int
) -> list[dict[str, str | int]]:
    if samples_per_question < 1:
        raise ValueError("samples_per_question must be positive")

    return [
        {"prompt": question, "sample_index": sample_index}
        for question in questions
        for sample_index in range(samples_per_question)
    ]


questions = [f"Solve for x: x + {n} = {2 * n}." for n in range(1, 1601)]
repeated = generation_requests(questions[:100], 16)
broad = generation_requests(questions, 1)

assert len(repeated) == len(broad) == 1600

Matching request counts isolates this allocation choice, although it does not guarantee equal token usage or training compute.

Why might repetition help? Fine-tuning includes the reasoning traces, not just final answers. Sathiamoorthy’s hypothesis is that several traces expose different ways to reason through the same problem. He also reports that stronger models were not invariably better teachers: teacher capability and the usefulness of its demonstrations to a student are different quantities.

Other results were similarly dependent on the recipe. Synthetic question generation and answering worked in the reported experiments, while answer filtering did not work well. The slide also lists question filtering as successful and increased source diversity as unsuccessful. These are experimental observations, not general prohibitions: the original paper explicitly notes that its unfiltered answer baseline contained twice as many examples as the filtered runs and was not compute-controlled. Filtering quality cannot be cleanly separated from that difference in training volume.

“Learnings” slide listing multiple answers per question, stronger models not necessarily being better teachers, successful synthetic question generation and question filtering, and unsuccessful source diversity and answer filtering.Open full source frame
OpenThoughts learnings on answer sampling, teacher models and data filtering.
10:5411:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:54 · section reference included

Extend the recipe to trajectories and environments

OpenThoughts-Agent extends the same experimental approach to agent trajectories and RL environments. The object being curated is now a task and its execution, rather than a reasoning answer alone. The scaling question remains: as the dataset grows, does the recipe continue to improve the trained agent?

Sources can include material such as Stack Exchange. From there, the decisions concern task mixtures, filtering, rollout generation and teacher selection. A rollout is the agent’s sequence of interactions while attempting a task, so choosing a teacher also means choosing the behavior the student will observe.

Several findings echo the reasoning-data work:

  • Teacher selection: stronger models were not necessarily better teachers. Sathiamoorthy tentatively recalls some Qwen models outperforming some Claude models as teachers, without identifying an exact comparison.
  • Repeated sampling: multiple samples helped again.
  • Synthetic task changes: rewriting and task augmentation did not work as well as expected.

The recurrence of these findings makes them useful experiments to run, rather than settings to adopt without measurement.

Sathiamoorthy reports that SFT supplied substantial gains, while compute-intensive RL contributed smaller final improvements in this work. His enterprise implication is practical: SFT can already work well for many applications, so the decision to add RL should depend on the remaining capability gap and the cost of closing it.

11:5412:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:54 · section reference included

Teach the form without memorizing the numbers

The enterprise example is Intuit’s Credit Karma, where the application explains why a credit card has been recommended. A prompted model can write the explanation, but compliance creates a second requirement. Long lists of rules increased latency without making the responses consistently compliant. Sathiamoorthy presents the engagement as a production post-training example.

Simply fine-tuning on existing examples introduced another failure mode: imbalanced values. If many examples contain 0% APR, the model can learn that recurring number and reproduce it where it does not belong. Fluent explanations are insufficient when a financial attribute is wrong.

The intervention changed the training representation. Instead of relying only on plain-language prompt-response pairs, the team added tags intended to make the model focus on the form of the explanation rather than particular numeric values. The talk does not specify the tag syntax or a runtime substitution procedure; the demonstrated principle is to change what the training examples encourage the model to learn.

Sathiamoorthy reports improved compliance, latency and throughput for the Credit Karma engagement, without numerical measurements or evaluation conditions. He also emphasizes ownership: a specialized model gives the enterprise more control over cost and reduces the pressure to revise its system whenever a frontier model changes.

13:4614:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:46 · section reference included

Turn prompts and logs into training examples

Curator provides tooling for the generation step. Inputs may come from a Hugging Face dataset containing prompts or from collected application logs. Generating responses to those inputs produces examples that can be used for fine-tuning. This makes the earlier curation decisions operational: the source prompts are inputs to a repeatable data-generation process.

Curator was used for the original OpenThoughts curation, and the talk describes integrations with Tinker and Fireworks. Those integrations belong to the later tooling: the repository dates them to March and June 2026, respectively, rather than the initial 2025 OpenThoughts release.

16:0416:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:04 · section reference included

Connect environment quality, execution and optimization

The closing architecture broadens the data pipeline into a stack for building environments and improving agents. Start with the environment layer: construct tasks, measure their quality and track their versions. Versioning matters because the environment is itself part of the training input, not merely infrastructure surrounding it.

Below that layer, compute and orchestration start sandboxes and generate rollouts. Long-horizon execution may require checkpointing, snapshots and rollback so that work can be preserved or an earlier state revisited. Above the environment layer sits agent optimization, including SFT and RL.

Optimization can also change prompts. GEPA uses LLM reflection to propose prompt improvements; Sathiamoorthy includes system-prompt and harness updates in this broader optimization layer. The closing stack therefore connects three responsibilities:

LayerResponsibility
RL environmentsBuild tasks, measure quality, track versions
Compute and orchestrationRun sandboxes and rollouts; preserve execution state
Agent optimizationImprove models through SFT/RL and prompts through reflection

The environment supplies the work, orchestration makes that work executable, and optimization uses the resulting experience to improve the agent. This is the architecture Sathiamoorthy proposes for both curating RL environments and post-training agents.

“RL Envs: What we are building” slide with three stacked boxes labeled “Agent optimization (post-training, GEPA),” “RL Envs,” and “Compute and orchestration.”Open full source frame
Three layers: agent optimization, RL environments, and compute and orchestration.
16:5217:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:52 · section reference included

Resources

From the talk

  • An Intuit and Bespoke collaborator explains product-scoped tags, controlled synthetic data and validation for credit-card recommendation explanations.

  • Installation instructions and examples for reflective optimization of prompts and other text artifacts.

Transcript download
Full transcript

Loading transcript…