Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face
AI Engineer World's Fair 2026 · 17:28
AI cybersecurity research and evaluation
Arithmetic develops cybersecurity evaluations and training data to improve AI models’ ability to discover vulnerabilities and reason through unfamiliar systems. Its work addresses a practical problem for automated defense: an agent can gather relevant information without understanding which actions that information makes possible. Arithmetic’s stated goal is to build model capabilities that help defenders outperform attackers.
Its Mask Off benchmark turns vulnerability research into interactive tests. Arithmetic’s researchers discover zero-day vulnerabilities in open-source software and build live environments containing connected applications. Agents investigate with tools, without internet or source-code access, to determine whether a user can perform an action beyond their assigned privileges. Deterministic checks measure intermediate progress as well as final success. This makes it possible to distinguish an agent that finds an authorization check from one that understands how inconsistent checks across services permit unauthorized access. The resulting behavioral data is intended to support model training.
Arithmetic also contributes cybersecurity expertise as a specialized data partner to GDPx, an evaluation of frontier-model capabilities across multiple domains. That participation places its security research within a broader effort to measure what models can accomplish on specialized tasks.
AI Engineer World's Fair 2026 · 17:28
Loading more talks and speakers…
Affiliations reflect their AIE event listings and recorded appearances, not necessarily current employment.