AI Engineer World's Fair 2026

Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face

Uri Rolls· CEO, ArithmeticThomas Wolf· Co-founder and CSO, Hugging Face17:28

Loading slides…

Read the talk

Training Frontier Models to Out-Think Hackers

Mask Off tests whether AI agents can reason through unfamiliar systems, exposing the gap between discovering an authorization check and understanding how it can fail.

From a talk by Uri Rolls and Thomas Wolf

Before you start: Familiarity with authentication, authorization, and AI agents that use tools will help you follow the examples.

A click changes something elsewhere

Click somewhere in a game, and something changes somewhere else. What did the action do? What state is the game now in, and which action should come next? The interactive puzzles in ARC-AGI-3 make this a concrete reasoning problem: a model must discover how an unfamiliar environment works by interacting with it. Thom Wolf opens with this example because cybersecurity can demand the same ability to infer rules, track state, and act on an understanding built during exploration.

Dark ARC-AGI-3 webpage with a pink handheld-style game panel, Start button, and mosaic of puzzle images.Open full source frame
ARC-AGI-3’s page pairs a pink game interface with colorful puzzle tiles.

An apparently simple interaction can expose a substantial capability gap. Wolf reports a 1–2% model success rate on ARC-AGI-3 in this discussion. His explanation is that strong models still struggle to build a dynamic model of what is happening in a world, even a small one.

Editor’s note: The ARC-AGI-3 paper defines an efficiency-based score relative to human action baselines. Wolf’s spoken success-rate figure should not be treated as a precisely specified percentage of games solved.

Arithmetic’s cybersecurity benchmark poses a related challenge: understand an unfamiliar environment, then act according to the model of that environment developed along the way. Wolf connects this research problem to a defensive purpose. He rejects a simple division in which closed models are good for cybersecurity and open models are bad; open models can also become part of the machinery that protects systems. The benchmark, developed with Uri Rolls, is intended to expose the capability gaps that still stand in the way.

0:451:15
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:45 · section reference included

Attackers gain reach; defenders still need coverage

The economic problem begins with an asymmetry. Securing a system resembles protecting a house: the defender must close every door and window, while an attacker needs only one overlooked opening. After entry, sensors provide another layer of defense. Rolls describes the existing cybersecurity stack as built around an important constraint on the attacker: limited resources force choices about which targets to pursue.

AI changes that constraint. Rolls describes models finding exploit primitives and zero-day vulnerabilities while defenders struggle to prepare for the resulting pressure. A skilled attacker using a powerful model can pursue multiple targets at once. Strong open models can create similar pressure. The change is therefore not confined to one provider or one distribution model; it concerns how much work an attacker can carry out with the resources available.

Defensive systems, meanwhile, must operate at scale with limited human intervention. Their effectiveness is constrained by what the models can do without constant expert guidance. Rolls briefly jokes about retreating to the woods, then returns to the practical alternative: make the models substantially more capable. Open models matter to that effort because the work requires collaboration.

Coding provides the motivating comparison. Better models have already changed what software agents can accomplish; Arithmetic proposes a similar capability-building path for cybersecurity. The slide pairs coding and cyber trajectories with high-quality evaluations and reinforcement-learning data. The goal is to produce defenders that outperform attackers, then rebuild the defensive stack around those capabilities. This is the proposed destination of the research, not an outcome established by the benchmark demonstration.

3:443:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:44 · section reference included

Where authorization rules disagree

Mask Off begins with a deliberately narrow scope: access control. Treating all cybersecurity as one benchmark would be like treating swimming, Formula One, and basketball as the same sport. Access control supplies a concrete objective: can a user obtain a foothold or perform an action beyond their assigned privileges? In Rolls’s example, he is an ML engineer and Wolf is an administrator at the same company. The question is whether the engineer can find a way to do something reserved for the administrator.

Broken Access Control ranks first in the OWASP Top 10:2025 list for web application security. Rolls describes the surrounding industry as roughly $30 billion. These failures remain a major concern despite years of attention.

The difficulty is often in the logic connecting systems. One component checks one property; another checks something different. Each local check may appear reasonable, yet their combination permits an unauthorized action. Finding such a failure requires understanding what the checks mean together, rather than merely recognizing a familiar bug in a piece of code. Mask Off aims to test that reasoning across the boundaries between components.

7:127:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:12 · section reference included

Unfamiliar environments with verifiable outcomes

The data begins with human vulnerability research. Rolls argues that finding cases outside familiar distributions still requires people to search for them. Arithmetic’s researchers discover zero-days in widely deployed open-source software, then use those findings to construct large, live environments containing several connected applications. The intended setup is unfamiliar to the model: it cannot inspect the source code, and the vulnerability comes from the team’s own research rather than a supplied description of a known exploit.

The agent receives basic tools needed to work on the task, but no internet or codebase access. It must learn from the behavior of the running environment. Because a complete solution can require a long chain of discoveries and actions, Arithmetic also uses deterministic checks along that chain. These intermediate checks make it possible to distinguish an agent that barely begins from one that gets close to the final objective.

The evaluated unit is the model plus its harness. Rolls credits Eugene from Anthropic for the evaluation graphic he adapts: task inputs based on real vulnerabilities feed an agent operating through tools, and graders evaluate the resulting behavior.

Evaluation elementWhat it establishes
Final binary graderWhether an underprivileged user accomplished an action they were not authorized to perform.
Deterministic intermediate gradersHow far the agent progressed through discovery and exploitation.

The two levels answer different questions. Final success establishes whether the task was solved; intermediate grading reveals which parts of the reasoning and action chain the agent could complete.

8:288:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:28 · section reference included

Finding the check is not enough

The walkthrough illustrates a real task rather than presenting a full technical disclosure. Its environment chains Keycloak, Vault, and a broker. The agent starts as a user with very limited privileges and must obtain access to production code. Each task has a reference solve script, providing a concrete successful path against which the agent’s behavior can be examined.

Rolls says the underlying zero-day was submitted to maintainers for verification. In his description, one authorization check uses a name while another uses an ID. The mismatch permits changing the real administrator’s name, inheriting privileges, and escalating access. Rolls describes the illustrated solution as a roughly sixteen-step chain. The identity-check mismatch is consequential because it enables progress through that larger sequence of actions across services.

The environment is too broad for an agent to test every possibility. The presented attempts by GPT-5.5 and Opus instead move among probes, trying different approaches. They reach the relevant check, but miss the inference that changing the administrator’s name could bypass the permissioning. This is a more specific failure than simply failing to discover the right endpoint: the agent encounters relevant evidence without deriving the action it enables.

Permission changes also alter other parts of a live system. An agent must therefore maintain and revise its understanding as it acts, just as a game-playing agent must account for a click changing something elsewhere. Discovery is not yet understanding: recognizing a check does not establish how changing the state will affect that check or the services behind it.

GPT-5.5 demo centered on Keycloak identity and roles, with realm-admin highlighted and a caption stating that the guard blocks the attempt and the model never renames the role.Open full source frame
The demo highlights a blocked attempt to obtain realm-admin by name.

The final requirement is to turn that understanding into exploitation code. A successful agent must reason precisely enough about the failure to implement and execute the necessary actions. The benchmark therefore connects exploration, state tracking, inference, and code generation in one task rather than treating a plausible explanation as a completed solve.

9:5810:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:58 · section reference included

Measuring the gap between discovery and exploitation

The live demonstration moves into Bach, Arithmetic’s internal evaluation orchestrator. Rolls presents a difficult benchmark snapshot using the labels K1 and K5:

ViewReported result
K1One solve in the displayed benchmark snapshot.
K5GPT remained the only public model reported to solve.

These are results from the presented snapshot, not general rankings of the models. With complete solves so sparse, partial grading becomes essential: it shows how deeply an agent penetrates the task even when the final outcome is failure.

The next view examines progress through a specific environment over time. On the specific environment displayed in Bach, GPT-5.5 is the only model shown making the decisive exploitation leap. This environment-level result is distinct from the earlier illustrated failed attempt. Other models discover nearly all the information they need, yet do not derive the exploitation that follows from it. The intermediate checks expose that gap between gathering evidence and using it.

Rolls’s defensive hypothesis is that making these reasoning leaps both reliable and fast across many models could give defenders a lasting advantage. The result view identifies the capability to improve; it does not itself demonstrate that defensive advantage.

12:1012:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:10 · section reference included

Three hours of activity can still miss the answer

A task called Falltime shows why traces matter alongside outcome scores. Rolls describes a displayed Falltime run that works continuously for more than three hours without solving the task. The duration reflects active work, not simply time spent waiting for code to finish. Long-running activity alone is therefore a poor proxy for progress toward the decisive inference.

The research work is to inspect what happened inside that activity. Rolls describes going through traces to understand what models did, why they did it, and where they failed. Intermediate graders identify the depth reached; the trace supplies the sequence of decisions needed to investigate a specific failure mode.

13:3413:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:34 · section reference included

Training for faster understanding

The desired next chart would show a model making the necessary capability leaps much earlier. The bright green future-model trace illustrates that ambition; it is hypothetical, not another measured result. Rolls connects faster understanding to faster defense, then proposes a collaborative training path built on strong open models. Those models could be post-trained for particular networks and environments, adapting their capabilities to the systems they would help protect.

Step chart with several model traces and a bright green trace labeled “Super cool future model” reaching the top near the left side.Open full source frame
The completed chart adds a hypothetical “Super cool future model” that progresses sooner.

Wolf closes the technical discussion by returning to the distinction between inspecting code and exploring a live environment. He contrasts Mask Off with a workflow in which a model reads source code to find vulnerabilities: here, the model encounters unfamiliar authentication infrastructure and must discover other services it needs to access and use. It begins without knowing how those pieces fit together. Fresh vulnerabilities and unfamiliar environments make exploration and understanding central research problems.

The proposed sequence is:

  1. Build benchmarks that make the capability gaps observable.
  2. Collect useful data about how agents operate in those environments.
  3. Fine-tune models and investigate how the resulting capabilities can protect systems.

Evaluation is the beginning of this process. The behavioral data is intended to support training, and the trained capabilities still have to be connected to defensive protection.

14:1814:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:18 · section reference included

The deployment constraint is speed

Once an attacker begins entering a system, the defender must recognize what is happening and stop it quickly. Wolf frames this as a race between attack and defense. A specialized model, perhaps running on specialized hardware, is one proposed way to shorten that response. He also warns against assuming that two major providers will solve the entire problem for everyone.

The alternative he proposes is to train defensive models, run them fast, and make them available to companies that need protection. That moves the discussion beyond benchmark capability to the practical availability and speed of a defensive system. It remains a deployment direction, with the reliability demands of cybersecurity making mistakes consequential.

Rolls ends by inviting collaborators to work directly on the data through a forum. Access control is only the first specialty; other areas of cybersecurity need comparable research and evaluation. Extending the work will require researchers with expertise in those domains, substantial post-training data, and models capable of turning that data into effective action.

15:4615:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:46 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…