AI Engineer World's Fair 2025

Safety and security for code-executing agents

Loading slides…

Read the talk

Safety and security for agents that execute code

Code execution lets agents solve more than programming tasks. Making that autonomy useful requires isolation, enforceable network limits, and review at the points where actions become consequential.

From a talk by Fouad Matin

Before you start: Familiarity with shell commands, Git pull requests, and basic container isolation will help you follow the implementation examples.

When solving a task means running code

What should an agent be allowed to do when the most effective way to answer a question is to run code? Fouad Matin approaches that question from building the systems that grant those capabilities. He describes spending six years running a security startup before joining OpenAI Security, then moving into Agent Robustness and Control in post-training and working on Codex and the open source Codex CLI. Those implementations supply the lessons here, with substantial safety work still ahead.

Frontier labs are improving coding benchmarks, but also the usability and deployability of agents. Matin’s prediction is that every agent will become a code-executing agent: writing software is only one reason to execute code; another is simply to accomplish a task efficiently.

Slide with a faded statement about frontier labs training models to write and execute code, followed by the emphasized conclusion that every agent will become a code-executing agent.Open full source frame
Every agent will become a code-executing agent.

o1 offered an early preview of reasoning-model capabilities. Matin describes o3 and o4-mini as more capable and reliable, shifting the deployment question from whether a model can perform an action to whether it should have permission to perform it. That question extends beyond software engineering.

Consider the multimodal example from the o3 release. An earlier model would inspect the supplied image and reason from that view. With code execution available, the agent can instead use OCR to decipher text or crop an image to examine a relevant region. Matin emphasizes that the prompt did not instruct the model to run code: the model selected the tool itself. A seemingly ordinary image question can therefore become a program-execution task without the user explicitly asking for one.

0:150:29
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:15 · section reference included

A simpler agent loop still needs a security boundary

Earlier agent architectures often put substantial software logic around the model: classify the task, load a specialized prompt and tool set, chain several loops, ask whether the work is finished, and use another model to produce the response. Matin argues that this orchestration is generally no longer necessary when a reasoning model can choose its tools and decide when to write and execute code.

Diagram contrasts a five-step agent inner loop with a simpler prompt-to-agent-to-code path. The words “do things” are crossed out and replaced by “run code.”Open full source frame
From complex agent loops to reasoning agents that run code.
DecisionExplicit orchestrationModel-directed loop
Task handlingClassify, then select a promptReason from the user’s request
Tool selectionLoad a task-specific setChoose among available tools
ExecutionChain predefined loopsWrite and run code as needed
CompletionSeparate checks and response stepLet the agent manage progress

The application can delegate more task logic to the model, but that delegation also gives the model a powerful execution capability. In security terms, Matin calls this remote code execution, or RCE. The same capability that makes the architecture simpler can backfire if its permissions are too broad.

The failure modes span both adversarial influence and ordinary mistakes:

  • Prompt injection and data exfiltration: untrusted instructions can redirect the agent and cause sensitive information to leave its environment.
  • Accidental unsafe changes: an agent can install a malicious package or write vulnerable code without intending harm.
  • Privilege escalation and sandbox escape: execution can cross the boundary intended to contain it.

Matin identifies prompt injection and exfiltration as priorities, but the other failures matter even when nobody is deliberately attacking the prompt.

2:292:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:29 · section reference included

Isolate execution, then decide where review belongs

Matin places these controls within OpenAI’s Preparedness Framework, describing safeguards against misalignment at large-scale deployment as a responsibility for both OpenAI and organizations adopting agents. The first practical safeguard is to give the agent its own computer.

In Codex in ChatGPT, that means an isolated container and a pull request at the end. Matin characterizes this separation as close to the safest practical arrangement. The important boundary is between the agent’s working environment and the environment where its output will eventually be integrated and run. For local execution, he lists containerization, application-level sandboxing, and operating-system sandboxing as ways to constrain the agent even if it attempts an unsafe operation.

Network access adds another route across that boundary. An agent might read documentation or a GitHub issue, encounter an injected instruction in a comment, and carry that untrusted content into the loop that executes code. If the loop also has access to a private codebase or other sensitive material, the consequences extend beyond an incorrect answer. Disabling or limiting internet access constrains both exposure to those instructions and opportunities to send information out.

Human review supplies another control point: inspect operations before execution, review final diffs in a pull request, or request confirmation for consequential actions. But a usable agent cannot require approval for every ls command. The design problem is to avoid both unrestricted execution and a stream of confirmations so routine that approval stops conveying meaningful judgment.

3:554:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:55 · section reference included

How the sandbox is built

A dedicated environment still needs enough capability to complete the task. Dependencies must be installed, and the agent must receive the access its work requires. Codex CLI provides an open source reference for builders who want to understand how local execution can be constrained rather than merely consuming a finished agent.

On macOS, Matin shows a portion of a Seatbelt policy. Seatbelt has been bundled with macOS since Leopard, but its documentation can be difficult to find. The team used models and deep research to investigate existing examples, drawing heavily on Chromium’s macOS sandbox design. A Seatbelt policy expresses permissions as rules; for example, the following fragment establishes a default-deny starting point and explicitly denies network operations:

scheme

(version 1)
(deny default)
(deny network*)

A usable execution policy must then allow the specific process and file operations the workload needs. The default-deny fragment alone is not a runnable development environment.

For Linux, Matin describes a Rust implementation developed with input from OpenAI’s security team. It combines seccomp and Landlock to provide unprivileged sandboxing intended to prevent escalation. The platform mechanisms differ, but the architectural role is the same: the system enforces a boundary around code execution instead of relying on the model to remember which operations it should avoid.

6:106:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:10 · section reference included

Allow useful work without allowing exfiltration

Matin describes the CLI’s then-current full-auto mode as a bounded execution environment: file reads and writes confined to the starting directory, and network calls available through commands approved for that access. Within that boundary, the agent can run pytest or npm test without repeatedly seeking permission. This is his description of the version presented, not a confidentiality guarantee for every Codex CLI version; the release labels and API names here retain the talk’s historical context.

For Codex in ChatGPT, Matin introduces internet access with configurable allowlists, describing it as a feature launched a day or two earlier. The Codex announcement dates that update to June 3, 2025. Configuration includes permitted domains and HTTP methods, accompanied by a security warning. This creates a policy choice between a tightly restricted environment and one with the connectivity required for a particular task.

Slide recommends disabling internet access and shows configuration controls for agent internet access, a domain allowlist, additional domains, allowed HTTP methods, and a security warning.Open full source frame
Disable or limit internet access unless the task requires it.

The reason for those controls becomes concrete with a request to fix a linked GitHub issue. The issue contains user-generated text instructing the agent to retrieve the latest commit and post it to an unrelated destination. The documented injection example uses git show HEAD output—commit metadata and a diff—as the payload sent to httpbin. The legitimate issue-fixing request has become a route for an attacker’s outbound-data instruction.

Matin attributes the compliance risk to strong instruction-following behavior: the agent may treat the issue’s contents as something to obey. Model training and suspicious-content detection can help it recognize that the embedded instruction should not be followed. But the system should prevent the outbound call even if the model attempts it. Denying access to the destination makes the network policy an authoritative control; combining that restriction with model-level defenses addresses both the decision and the ability to act on it.

7:468:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:46 · section reference included

Review what will cross into a privileged environment

More generated code means more code to review. Automated pull-request review and language-model assistance can reduce the burden, but Matin does not treat them as substitutes for a person reviewing the operations an agent is about to perform. A particularly consequential case is a newly introduced dependency: an unfamiliar package, or a name differing from the intended package by one character, can enter the codebase during an otherwise plausible change.

The risk can outlive the agent’s sandbox. A malicious dependency introduced during restricted execution may later run in a privileged environment. Review therefore needs to consider what the change will do after integration, not just whether the agent’s immediate operations were contained.

The same principle applies outside coding. Matin uses Operator as an example: a domain list and an in-loop monitor help identify potentially sensitive actions, while its monitoring task and watch mode keep a human reviewing those actions. The objective is selective supervision where consequences warrant it, while preserving useful freedom elsewhere.

9:4910:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:49 · section reference included

Give the model tools and enforce the deployment boundary

With those boundaries in place, more of the agent’s task logic can move into the reasoning model. Matin presents an execution tool exposed as local_shell in the API, describing it as matching how the models were trained to write and execute code. He also introduces apply_patch: models struggle to produce correct line numbers for conventional diffs, so a dedicated patch format gives them another way to express file changes. MCP and web search complement these execution tools.

A concrete combination is Socket’s MCP server, which lets an agent query dependency risks before installing a package. Matin describes two possible places for the check:

  • Inside the agent’s workflow: the agent consults Socket while deciding whether to install a dependency.
  • At a system checkpoint: the surrounding application checks proposed dependencies after the agent’s rollout, before allowing installation.

Socket’s contemporary MCP release was experimental and supplied dependency risk scores. It can help identify suspicious or vulnerable packages; it does not certify that a dependency is safe. The distinction between the two placements is who ensures that the check happens.

For execution itself, Matin again recommends a remote container. He announces a container service associated with the Agents SDK and Responses API, and describes the deployment choices as running locally, in an organization’s own environment, or with OpenAI hosting execution. These are the options presented in the announcement, with isolation remaining the central design requirement.

Diagram connects a prompted agent to Tools and Exec branches. Tools connects to a Socket MCP server labeled as checking vulnerable dependencies before installing; Exec offers a local shell or remote container.Open full source frame
Using Socket through MCP to check dependencies before installation.

The operating tradeoff is how much autonomy the environment can safely support. A container or OS sandbox with networking disabled can let an agent work for longer without interruption. Reading external documentation and installing packages require more connectivity, so those permissions should be granted deliberately for the work that needs them. Human review remains part of that arrangement: Matin sees language-model monitors as valuable, but not yet able to provide the certainty of deterministic controls.

The remaining work spans both sides of that boundary. Matin closes with plans for more tooling in the Codex repository and documentation on ML interventions and system controls, alongside recruiting for Agent Robustness and Control and Rust work on Codex CLI integrations. More capable models can choose how to accomplish a task; the surrounding system still has to determine what they can execute, where information can go, and when a person must take responsibility for the result.

11:0311:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:03 · section reference included

Resources

From the talk

  • Source repository for the terminal coding agent, with installation instructions and implementation resources.

  • Introducing CodexArticle4:33

    Historical introduction to Codex's isolated cloud tasks, review workflow, and subsequent June 3 internet-access announcement.

  • Introduces Socket's experimental MCP server and explains dependency risk queries through its depscore tool.

  • Introducing OperatorArticle10:32

    Historical introduction to Operator and its human oversight mechanisms, including watch mode for sensitive sites.

  • Explains Chromium's use of macOS Seatbelt to restrict process privileges.

Updates since the talk

Transcript download
Full transcript

Loading transcript…