Stop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI

Loading slides…

Read the talk

Stop Giving Your Agent Tools. Give It a Sandbox

Mikyo King explains how Arize Phoenix uses code mode to discover tools, compose operations, and keep intermediate data out of model context—and why the savings come with new security, reliability, and evaluation work.

From a talk by Mikyo King

At a glance

Ideas worth remembering

  • Code mode combines tool discovery with a programming environment. Programs can compose multiple calls and filter intermediate results before anything returns to model context.

  • Phoenix applies the idea through Bash and files, Python-accessible REST operations, SQL aggregation, and a Web Worker programming interface bound to front-end state.

  • A sandbox does not settle secrets, permissions, network access, recovery, or artifact transfer. Composed programs and nested execution environments introduce their own failure modes.

  • Arize's reported Harbor comparison used 43% fewer tokens and cost 31% less, with approximately unchanged latency. Repeated task verifiers and captured trajectories support evaluating behavior rather than relying on one successful run.

  • A new tool can initially make an agent less efficient. Inspect the affected trajectories, identify instructions that steer it poorly, and use task evaluations to guide changes.

An inbox task reveals the value of writing software on demand

Getting every invoice out of an email inbox is a small task with two very different workflows. A programmer can write a script that checks messages for attachments, downloads them, and puts them in a folder—perhaps fifty lines of code in Mikyo King's opening example. Someone restricted to the application's interface has to open messages and download the files manually. The difference is the ability to create a missing operation rather than repeat the operations the interface already offers.

King, who heads the open-source group at Arize AI, uses this example to introduce the team's work on agents inside its observability and evaluation platform. Coding agents such as Claude Code make that scripting capability available through a request: the agent can build a small piece of software for the task. His architectural question follows from that change. If a model can write the operation it needs, why design an agent that can only select from a fixed menu?

0:421:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

The classic tool loop makes context carry the work

A conventional tool-calling agent starts with a catalog of available tools. A user asks a question; the model chooses a tool; its output enters the context window; the model reasons over that output and chooses another tool. King explicitly acknowledges that this works well for a bounded set of capabilities. The strain appears when the catalog grows and tasks require more data and longer sequences of actions.

Two kinds of material accumulate: descriptions of tools that may be irrelevant to the current task, and results from operations the agent has already performed. Pagination makes the latter especially costly because retrieving a dataset can require repeated calls and repeated additions to context. More turns and more accumulated material eventually force compaction. The model's working context becomes both a place to reason and a transport for intermediate data.

Repeated context-full diagram with a red-outlined context window filled by tool definitions, a user question, tool calls and results, beside a model and six MCP servers.Open full source frame
The CRM, meetings, email, calendar, docs and tasks servers all contribute tool definitions. Each subsequent tool call and result takes up more of the same context window.
1:422:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:42 · section reference included

Code mode moves composition and filtering into an execution environment

Code mode gives the agent two capabilities: finding the tools relevant to its task and executing them inside a programming environment. The program can call operations in sequence, run independent calls in parallel, and put conditions around subsequent work. Multiple tool operations can therefore happen within one model turn. Before returning, the program can select the part of the result the model actually needs.

Where do intermediate results go? In the classic loop, tool outputs return to model context for the next reasoning step. In code mode, the execution environment can receive and process those outputs first, then return a smaller result. Tools still do work, but each intermediate result no longer has to become model-visible text. Fewer model turns and less returned text are the intended savings, rather than a promise that tool execution itself becomes free.

A model sends a program to a sandbox that calls meeting, transcript, deal lookup and attachment tools; the attachment line is highlighted.Open full source frame
The program finds the Acme meeting, gets its transcript and attaches it to the deal. The only value it sends back to the model is “attached.”

King separates three mechanisms that make this architecture useful:

  • Progressive disclosure: Search for a capability, learn how to use it, and load only the tools needed for the task instead of carrying the whole catalog upfront.
  • Composition: Use ordinary program control flow to chain dependent operations or fan out independent ones. The model does not have to mediate every individual call.
  • Selective return: Search, trim, or otherwise process tool outputs in the execution environment, then send only the useful portion back into context.

The fourth mechanism is the one Arize particularly values: creating a missing tool. Consider a request to remove personally identifiable information from a set of records. If the fixed catalog has no redaction operation, the agent either lacks the capability or tries to transform the records in its own context—bringing sensitive values such as credit card numbers into that context along the way. A coding environment lets it write a transformation that runs on the records before returning the needed output. This is a capability the architecture enables, not a guarantee that generated redaction code is correct or that sensitive values cannot escape.

Where intermediate tool results go

Model contextClassic tool calling — Contains tool descriptions and accumulated tool outputs; the model chooses subsequent calls.
Individual tool callsClassic tool calling — Each returned output enters context before the model reasons about another call.
Model contextCode mode — Uses relevant tool information and receives the selected result of a program.
Programming environmentCode mode — Runs sequences, parallel calls, and conditions; searches or trims intermediate outputs.
Discovered toolsCode mode — Perform the operations composed by the program.
The classic loop returns tool output directly to model context. Code mode inserts a programmable execution environment that composes calls and selects what returns to the model. Arrows distinguish execution requests from returned data.
Read the diagram as text
  • Model context. Classic tool calling — Contains tool descriptions and accumulated tool outputs; the model chooses subsequent calls.
  • Individual tool calls. Classic tool calling — Each returned output enters context before the model reasons about another call.
  • Model context. Code mode — Uses relevant tool information and receives the selected result of a program.
  • Programming environment. Code mode — Runs sequences, parallel calls, and conditions; searches or trims intermediate outputs.
  • Discovered tools. Code mode — Perform the operations composed by the program.
  • Model context → Individual tool calls: Requests an individual operation.
  • Individual tool calls → Model context: Returns tool output into context.
  • Model context → Programming environment: Supplies code that composes operations.
  • Programming environment → Discovered tools: Executes calls using program control flow.
  • Discovered tools → Programming environment: Returns outputs for processing.
  • Programming environment → Model context: Returns only the needed result.
3:123:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:12 · section reference included

Phoenix adds Bash, Python, SQL, and a programmable UI

The agent and harness inside Phoenix began with Bash. Shell utilities such as sed and grep could be composed with Arize's own CLI, and intermediate outputs could be written to files for later use. Files give the program somewhere to keep working material without requiring every byte to travel through the conversation.

A Python sandbox expanded this approach by exposing Phoenix's REST create, read, update, and delete operations inside the coding environment. Getting traces and manipulating datasets became operations a program could combine. The change was in how the existing capabilities could be used: instead of asking the model to coordinate a separate turn for every operation, code could compose them.

SQL addressed a different limitation. A catalog of record-level operations is an awkward interface for an aggregate question about LLM spans and P95 latency. Retrieving page after page of traces just to compute an aggregate moves too much data through the agent. Allowing the model to write a query lets the database perform the relevant aggregation and selection. King's example uses placeholder latency conditions rather than a concrete query, but the distinction is clear: query the population where it lives instead of fetching the population to answer the question.

The next example asks a more unusual question: how can generated code operate the interface the user is already viewing? Phoenix provides a UI-bound programming interface that runs in a Web Worker with two-way data binding to the browser UI. The agent can change the time range and read from the front-end data store. This connection allows code to produce visible UI changes and access UI data, yielding what King describes as computer-use-like behavior through a programming interface. His delight is part of the point: it is fun to watch it “cook.”

6:116:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:11 · section reference included

The sandbox shifts complexity into security and operations

“Code mode is not … for free” is King's transition from capability to responsibility. Executing model-written programs means managing CPU, memory, and how long hosted environments remain available. Network access adds another decision: an agent may need the web, but outbound access exposes the environment to destinations and content the application does not control. Choosing a sandbox does not settle its resource limits or network policy.

Two security questions remain distinct:

  • Secrets: Code may need credentials to use a service, but those values must not return to model context. Vaulting and the return path both matter; enabling access is only half the problem.
  • Permissions across a program: An authorization can fit the first five operations and be wrong for the sixth. Composing individually available operations can also create a permission problem. Guardrails therefore have to account for what the composed program does, rather than treating access to the environment as sufficient approval.

The operational failure modes also change:

  • Startup failures: Sandboxes do not always boot successfully, so the harness must cope with environment failures before useful execution begins.
  • Program failures: Debugging now means understanding generated code, not just inspecting one tool call. The tool's parameter is effectively a program, and it can fail halfway through its work.
  • Discovery failures: Removing the upfront catalog creates a need to discover and understand available capabilities. That can require an additional handshake before the agent knows what it can call.

A less obvious cost appears when both an MCP integration and the surrounding agent harness implement code mode. Results from one sandbox may be serialized as JSON, parsed, and then brought into the other coding environment. King calls this the double-sandboxing problem: two layers each provide execution machinery, while data still has to cross between them. Adding code mode independently at every layer can recreate transfer and integration work that the architecture was meant to reduce.

8:128:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:12 · section reference included

Longer programs need recovery, artifact handoff, and capable models

Batching more work into a program makes recovery more consequential. If a large generated program breaks halfway through, a sandbox alone does not provide the ability to resume from the failed step. King identifies durability as an open problem rather than a blocker. He also flags determinism and mathematical requirements as reasons to consider the execution language: a language a model writes fluently may still be a poor fit for a particular numerical requirement. The talk does not develop a specific numerical failure or language comparison.

Images expose another return-path problem. Code can manipulate an image or create binary data inside the environment, but serializing the entire image into model context defeats the selective-return design. King suggests some form of signed image handoff so the artifact can leave the sandbox without turning its bytes into conversation content. This is a proposed approach, not a completed handoff protocol described in the talk.

Model choice constrains the economics too. King reports strong code-mode capabilities from frontier models, while warning that a smaller replacement may not be capable enough. Reducing tokens is one source of savings; substituting a cheaper model is another, and the first does not establish that the second will work. Code mode still depends on the model being able to write and manage the required program.

10:4311:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:43 · section reference included

Harbor trials measure task success alongside token and cost savings

Arize evaluates its own agents using Phoenix and the Harbor Framework. In the described setup, Harbor places agents in sandbox environments, gives them clear task instructions, and turns network access off. Multiple trials and different configurations help distinguish repeatable behavior from a run that happened to work.

Harbor verifiers check whether the task was performed and assign a binary result: one or zero. The team tracks pass@k for individual tasks across repeated attempts. That matters because an agent can take different trajectories on the same task; a single successful run says little about how consistently the system reaches the required outcome.

What changed in the pre/post code-mode comparison? King reports 43% fewer tokens and 31% lower cost, while latency stayed approximately constant despite sandbox overhead. He describes the work as spanning Claude Code, MCP, and CLI interfaces. The figures apply to Arize's reported Harbor run. In his spoken comparison, King gives no task count, model configurations, or numerical before/after success rate.

Dashboard comparison showing 43% fewer tokens, 31% lower cost, unchanged reward, and a 0.6% latency increase with code mode.Open full source frame
King reports that PXI used 43% fewer tokens at 31% lower cost in the Harbor comparison. Reward stayed the same and latency rose 0.6%.
11:4412:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:44 · section reference included

Use trajectories to explain why a new tool makes things worse

The useful output of evaluation is more than a score. Capturing the agent's trajectory alongside the verifier result lets the team inspect how it reached that result. King gives a counterintuitive example: introducing a new tool can initially reduce efficiency. The additional capability changes the agent's behavior, and its instructions may steer it toward a worse path in particular scenarios.

Those scenarios provide something concrete to fix. The team can show a coding agent which parts of the instructions guided behavior in the wrong direction and use the evaluations to guide subsequent changes. The verifier checks the outcome; the trajectory helps explain the route. Together they make it possible to improve a harness based on observed behavior rather than assume that more tools must make it better.

King closes with Arize's reasons for adopting code mode: fewer turns, lower cost, and room for agents to create capabilities and perform longer-horizon tasks. Supporting a user's choice of model remains an ambition alongside the capability limits he has just described. His practical invitation has two parts: give the agent a sandbox, then evaluate what it actually does there.

13:1413:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:14 · section reference included

Resources

From the talk

  • A practical starting point for tracing agent behavior and running evaluations with the platform used in the talk.

Transcript download
Full transcript

Loading transcript…