AI Engineer World's Fair 2026

Adaption Labs — Gradient-Free Continual Learning

Sara Hooker· Co-founder, Adaption20:51

Loading slides…

Read the talk

Adaption Labs — Gradient-Free Continual Learning

Automating training recipes and changing where compute pays off could widen participation in frontier AI, but data quality, model size, safety and access still shape the opportunity.

From a talk by Sara Hooker

Before you start: Basic familiarity with model training, inference and hyperparameters will help; no knowledge of AutoScientist is required.

The narrow path into discovery

Who gets to ask questions at the frontier of AI—and who first has to earn access to the tools? Sara Hooker opens with that problem. Computer science is young enough that she describes its history as roughly two generations of researchers, yet the institutions and subjects that define its frontier have already changed dramatically. Her wartime framing needs one historical distinction: the first transistor was demonstrated in 1947, after World War II, and Manchester’s experimental transistor computer followed in 1953.

The longer history of science offers a parallel. Wealthy independent investigators could once pursue discovery as gentleman scientists. Scientific associations, including the Royal Society in the seventeenth century, helped establish a more organized research community; full-time scientific work became a profession. Institutions made sustained discovery possible, but they also created gates around participation. Hooker borrows Rosanne Liu’s phrase, “the unreasonably narrow path,” to describe the result.

Her displayed example is Yann LeCun’s career, with a connected sequence running from a PhD and postdoc through AT&T Labs, NEC Labs, NYU and FAIR. The general prescription is familiar: enter the right PhD program, move into the right industry lab, produce work that the institution considers sufficiently interesting, and eventually contribute at the frontier.

Slide showing connected stages from PhD to postdoc, AT&T Labs, NEC Labs, NYU and FAIR beneath the heading about barriers to entry.Open full source frame
The professionalization of science introduced barriers to entry, illustrated through Yann LeCun’s career path.

Hooker places her own efficiency-at-scale research, PhD and work at DeepMind within that filtered system. A famous advisor and important internships help; curiosity about the wrong problem at the wrong time can leave someone outside it. Yet learning to represent the world computationally was never the ultimate purpose. The purpose was to answer the questions that those representations make tractable. Access to the tools determines access to the questions.

0:180:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:18 · section reference included

Compute adds a second gate

AI compounds the institutional filter with a hardware filter. Hooker points to Merve’s GPU-poor versus GPU-rich tweet, then to anonymized companies A, B and C: a handful of labs with the resources to build the models everyone else uses. A map she attributes to Stanford illustrates geographic concentration. The relevant distinction is between concentrated model development and discovery as a whole: Stanford’s AI Index tracks notable model production, while also recognizing academia’s continuing strength in highly cited research.

That concentration affects the product delivered downstream. As model development shifts toward industry, a common model is shipped to people with very different needs. Uniform delivery also risks uniform spending: an easy task and a hard task need not deserve the same compute budget. Broader participation therefore requires more than permission to use a frontier model. Builders need control over how intelligence is customized and where computation is spent.

3:073:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:07 · section reference included

Automating the data and training loop

AutoScientist addresses the expertise barrier by automating model training. The intended starting point is a domain question, rather than years spent acquiring the training knowledge needed to investigate it. Hooker describes a loop that spans data through alignment and changes its choices according to the domain and type of data.

Hooker reports that AutoScientist outperformed research-staff configurations in Adaption’s internal evaluations across model sizes and architectures, including dense and mixture-of-experts models. Her explanation is about search breadth: individual researchers tend to know particular model families well, whereas an automated process can explore a wider configuration space and accumulate experience. This is a comparison of training configurations in those evaluations, not a measure of general scientific ability.

The gains depended on co-optimizing data and model choices. Giving an agent permission to generate data was insufficient. The system needed to control data quality and choose model adaptations in light of the changes made to the data. Knowledge gained while adapting the dataset then informed the training process, alongside domain knowledge and experience from other components within that domain. Data preparation and model optimization were coupled decisions, not independent stages whose outputs could simply be handed over a wall.

The stopping condition matters when interpreting the results. Hooker says the displayed win rates above 60% reflected a budget rule: the agentic flow could exit once it exceeded 60%. She reports further improvement after that threshold was removed. The clustering therefore reflected when experiments were allowed to stop, rather than an identified ceiling on performance.

A small Python function makes that historical threshold precise. With win rate represented as a fraction, reaching exactly 0.60 does not satisfy an above-60% condition; 0.61 does. Removing the target leaves the separate iteration budget in place:

python

def should_stop(
    win_rate: float,
    iteration: int,
    max_iterations: int,
    target: float | None = 0.60,
) -> bool:
    target_exceeded = target is not None and win_rate > target
    budget_exhausted = iteration >= max_iterations
    return target_exceeded or budget_exhausted

print(should_stop(0.60, 3, 10))  # False
print(should_stop(0.61, 3, 10))  # True
print(should_stop(0.61, 3, 10, target=None))  # False

This illustrates the stopping decision, not AutoScientist’s implementation. The separate iteration limit is a teaching budget; the historical behavior Hooker identifies is permission to exit after exceeding the win-rate target.

The search also changes many hyperparameters together. Human researchers are often cautious about simultaneous changes, which makes experiments easier to interpret but limits how aggressively they explore combinations. Hooker sees broader configuration search as a route to more predictable customization: use domain knowledge, find a productive recipe, and reduce training compute spent on unsuccessful attempts.

4:595:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:59 · section reference included

From domain customization to interaction

The beta had been announced roughly four weeks before the talk. Hooker describes particularly strong interest from medicine and science, alongside legal and code applications. These are domains where a general model can fall short and the final layer of customization matters greatly. The displayed domain-distribution chart puts Medical and Science at the top, with the longest bars.

Horizontal bar chart titled “Domain Distribution Across All Datasets,” with Medical and Science as the longest bars, beside text naming medical, science, legal and code as domains where general models fall short.Open full source frame
Medical and science lead the displayed domain distribution, alongside demand in legal and code.

The immediate goal is to shorten the innovation cycle and improve the likelihood that a training run produces a useful result. The next extension, Hooker says, is adaptive test-time compute: choosing inference effort according to the task. That is future work in this account, rather than a capability demonstrated here.

Accessibility also includes language and task coverage. Hooker says the team chose to cover 242 languages from day one. That is a coverage commitment, not a claim of equal model quality across every language. She also emphasizes non-verifiable tasks: everyday work whose quality cannot be reduced to a straightforward automatically checked answer. Such tasks are central to the learning problem, even though they lack the convenient feedback available from a passing test or a correct numerical result.

This changes the unit of design. In the older organizational model, one team built a monolithic model, another served it, and another owned the frontend. An interacting system makes those boundaries less adequate: how efficiently it acts, what it learns from the environment, and how that learning affects subsequent behavior become central concerns. The mandate is continuous adaptation through interaction, with the training loop as one part of that larger system.

7:428:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:42 · section reference included

Where compute pays off is changing

Automating training expertise would not solve access if every user still needed an enormous GPU cluster. Hooker makes the problem concrete with a hypothetical offer: use the agent, but bring ten thousand GPUs. Removing the expertise barrier only changes participation substantially if the compute barrier changes too.

Her second argument, associated with On the Slow Death of Scaling, is that increasing pre-training model size is no longer the most rewarding use of additional compute. The distinction has infrastructure consequences: tightly coupled pre-training favors large, co-located clusters, while inference and some other uses of compute allow more distributed execution.

Compute allocationInfrastructure implicationOpportunity Hooker emphasizes
Larger pre-training runsCo-located capacity, with redundancyReturns constrained by the architecture
Inference and other computationMore scope for distributed capacityBetter returns from how compute is used

Hooker attributes weakening returns from pre-training size to architectural saturation and argues that other allocations can deliver more performance per FLOP. These are her claims about the changing optimization landscape, rather than a quantified comparison supplied for a particular workload.

Hooker describes an Open LLM Leaderboard comparison in which the daily best model under 13B parameters overtakes larger models over time. The comparison’s benchmark aggregation and dates are not specified here, so it supports her discussion of changing relative performance without establishing that smaller models win on every task. She also argues that recent size increases have not reproduced the stepwise gains of earlier generations.

The resulting opportunity is a broader optimization space. Performance can depend on recipes, algorithms, data decisions and the actions surrounding a model, rather than predominantly on enlarging a pre-training run. Those are places where research and automated discovery can matter without requiring ownership of the largest cluster.

That brings the prepared argument back to the opening question. If acquiring the tools becomes less expensive and less institutionally constrained, the research question itself can become the starting point. The promise is not that scientific judgment disappears; it is that more people can apply it before passing through the prescribed career funnel.

10:1310:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:13 · section reference included

How broader participation would happen

The first audience question asks for the mechanism behind democratization. Hooker estimates that fewer than 5,000 people worldwide know how to train frontier models at scale. She characterizes much of that expertise as configuration knowledge passed between apprentices: valuable, scarce, and well suited to automated exploration.

Two effects would work together:

  • Cheaper experimentation: Automating training decisions shortens innovation cycles. Lowering the cost of asking a research question changes both the number of questions people can explore and which questions they consider worth asking.
  • Different compute purchasing: Agentic and post-training computation contribute to performance without necessarily demanding the same accumulation of co-located GPUs as large pre-training runs. That changes the relative advantage of infrastructure ownership and a productive idea.

The second effect does not eliminate compute costs; it changes the resources a participant needs to compete.

13:4913:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:49 · section reference included

Adaptability also changes the risk

The next question exposes the other side of user control: someone could take a model designed to behave safely and adapt it into an unsafe one. Hooker acknowledges that wider tool access, including open source, carries real risk. Greater availability changes what people can do with a system, including harmful uses.

She separates customization from open-source release. AutoScientist is intended to let users control their own models, whether locally, privately or within a company. Those deployment settings do not by themselves determine whether the system or its models are openly released.

Restrictions also have a cost: they decide who can participate. Hooker rejects both the claim that openness is risk-free and the assumption that restricting access resolves the issue without other consequences. Her answer identifies the trade-off rather than proposing a technical enforcement mechanism for safe adaptation.

15:2015:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:20 · section reference included

Optimizing the model and its harness together

Asked which research directions are most promising, Hooker identifies the balance between parametric storage, held in model weights, and non-parametric storage, held outside them. An adaptive system must decide what belongs in each. That makes learning a problem of placement as well as acquisition.

She then extends the data-and-model lesson to the harness, the surrounding system that organizes a model’s interactions. An AutoScientist for harnesses would need to optimize the harness together with the model. A choice that helps one component in isolation may not be the best choice for their combined behavior. Hooker describes this as a long-horizon problem involving joint choices and co-training, rather than an independent search for the best wrapper around a fixed model.

16:5917:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:59 · section reference included

Large models still matter

A further audience question challenges the compute argument directly. Reinforcement learning and fine-tuning remain GPU-intensive, particularly for large models. Smaller frontier models also often learn from larger teachers through distillation. If their capabilities depend on expensive upstream models, how far does the apparent reduction in size really broaden participation?

Hooker accepts that frontier models remain large. Her claim is narrower than saying size no longer matters: Hooker predicts that frontier labs will not increase pre-training model size another 4× under the current architecture. A new architecture could change that ceiling. She is not recommending that everyone use a 0.8B model; she expects a more level competitive field among builders working near the existing upper end.

Distillation remains useful because better data lets a model use its capacity more effectively. Hooker also expects techniques associated with post-training to move earlier into pre-training, making the quality and organization of learning a stronger lever. Very large models may still deliver benefits on a small, long-tail portion of the task distribution. The unresolved engineering question is whether those gains justify the additional pre-training compute.

17:4418:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:44 · section reference included

Removing the immediate compute hurdle

The final question asks about plans to open source model settings. Hooker answers by pointing to the beta and saying that GPUs are being offered for free to remove the compute hurdle. That is the concrete access offer made in the talk, not an explicit open-source commitment or a promise of permanently free compute. The ending puts the participation argument into practical terms: make it possible for someone with a domain question to begin an experiment.

20:0620:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

20:06 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…