AI Engineer World's Fair 2026

Designing Agents (The Floor Is the Frontier)

Ben Hylak19:46

Loading slides…

Read the talk

Designing Agents: The Floor Is the Frontier

Reliable agents need more than impressive capabilities: they need tests of the whole system, production issues tracked over time, and clear boundaries between detection and investigation.

From a talk by Ben Hylak

Before you start: Familiarity with tool-using agents, execution traces and basic software tests will help; the code example uses TypeScript.

What actually makes a deployed agent better?

Where does evaluation advice stop helping with the agent you are actually building? That is Ben Hylak’s opening question: which parts of the workflow remain annoying, unreliable or poorly served by the prevailing advice? He observes that continual learning is still uncommon in deployed products and labs, despite the attention it receives. There is no settled, one-size-fits-all method for improving agents, so he frames the session as a practical discussion and invites questions from builders.

The earlier chatbot problem was easier to specify. Ask for the capital of the United States, expect Washington, D.C., and check the answer. Hylak roughly estimates that developers already knew the answers to 80–90% of the questions their chatbot users would ask. That made fact checking a plausible center of an evaluation strategy.

Agents operating in finance, healthcare and defense introduce a different problem. Hylak initially disliked the term agent, but came to appreciate the distinction it draws: these systems use tools, encounter obstacles and improvise ways around them. An agent might decide to decompile something to get past a roadblock—a workaround its developer never anticipated. The same flexibility can produce an ingenious solution or a harmful action. Evaluating the final answer alone no longer captures everything that matters.

Slide contrasting agents a year ago with chatbots becoming agents, with a smaller line mentioning finance, healthcare, and defense.Open full source frame
“A year ago, agents barely existed. Now, every chatbot has become an agent.”
0:150:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:15 · section reference included

When the harness changes, what does the test still mean?

Much evaluation advice still assumes the chatbot setting: assemble a predefined dataset, then grade responses against expectations. Hylak uses a 1,000-example dataset as his example of that prescription. The problem appears when expectations encode the implementation rather than the behavior the product needs. A test might require a particular tool call whenever the user asks a particular question; replacing the model or harness can invalidate that expectation.

White slide reading: “Meanwhile, agent evals are stuck in the chatbot era. Pre-defined, fragile test-cases, etc.”Open full source frame
Agent evals remain stuck in the chatbot era of predefined, fragile test cases.

In Hylak’s illustrative harness-switch example, moving to Claude Code CLI makes 80% of the existing tool-specific evals unhelpful. This is a hypothetical illustration, not a measured failure rate. The underlying concern is durability: if the agent has a different set of tools or a different way to solve the task, a test that insists on the old route may no longer tell you whether the new system works.

That maintenance burden becomes a product decision. After spending months constructing an eval set, would a team delay adopting a new model by two weeks to update it? Hylak suspects most would not. His criticism of what he calls “big eval” is directed at this gap between prescribed process and actual release decisions. Evaluation should buy safety that teams will use, rather than create an elaborate gate they bypass whenever it becomes inconvenient.

Hylak’s perspective comes from his work as Raindrop’s CTO and co-founder. He describes the company’s work as finding critical production issues, checking that fixes work without unexpected side effects, and simulating changes against past behavior before release. He names Vercel, Speak and Framer as customers and reports adoption among Fortune 100 companies. That exposure informs advice the company revises as models and effective practices change.

Its open-source tracing tool, Workshop, also provides a place to experiment with self-healing loops: when a needed capability is missing, an agent can extend the tool itself. Hylak reports that Workshop has thousands of users. He compares hosted Raindrop’s agent issue detection to Sentry and points to How to evaluate AI agents as the company’s practical guide to these workflows.

3:574:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:57 · section reference included

The product team’s responsibility

The useful question is not simply which issues an agent has, but which changes will make it better with the tools available. Some problems cannot yet be solved economically. Hylak uses Fable as an example: training a comparable model yourself may make less sense than waiting to integrate it. This leads to his distinction between a benchmark maxer and a floor raiser.

The word eval obscures that distinction because it can mean a lab benchmark, an application test or an online assessment. Borrowing a model lab’s language does not give a product team the same job or the same tools.

ConcernModel labApplication team
Main targetGeneral-purpose capabilitiesA particular product’s behavior
Knowledge to supplyBroad model competenceCompany data and domain meaning
Integration problemModel behavior at the APIData shapes, access and tools

Application teams must teach the system what their data means and how to access it. Their responsibility extends into the environment around the model.

User expertise changes that responsibility further. The guide’s quiz asks whether users are domain experts and whether the product augments or replaces their work. A developer can delete an incorrect Copilot autocomplete or Cursor tab completion. Engineers using Claude Code CLI or Codex can also recognize mistakes and repair their environment; a failure might originate in something installed incorrectly on the user’s computer. Products such as Devin shift more work onto the system. An AI doctor raises a different question again: how much can the user reasonably be expected to detect and correct?

The ceiling is the most impressive capability the agent can exhibit. The floor is its worst behavior: recommending a competitor, deleting data, or sending an unwanted AI-written email to a customer because it happened to have email access. These failures damage trust even when the system is otherwise highly capable. Hylak cites GPT-4o sycophancy as an example of a floor problem rather than a missing capability. Raising the floor means attending to the behaviors that make users unwilling to trust the product.

7:107:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:10 · section reference included

Test the whole agent, locally

Offline evaluation needs to follow the shape of the system being evaluated. A string.includes() check on an answer, or a playground for editing a cloud-managed prompt, captures only a narrow slice of an agent. Hylak says he knows only one or two companies still using managed cloud prompts. In the systems he sees, the effective prompt includes the code, the harness and everything connected to it—not just an instruction string.

Keep evals as code and make them look like software tests. Unit tests can exercise individual components; end-to-end tests can exercise the agent through its harness. Hylak cites Sentry’s Vitest Evals, which he describes as Vitest with syntactic sugar, and invokes OpenAI’s term macro evals. His practical recommendation is straightforward: run tests on the agent locally, alongside the code that determines its behavior.

Slide with guidance to make evals code-aware and think in terms of software testing, alongside a dark code example and a note to keep evaluation local.Open full source frame
Evaluate the agent, not an LLM call.
11:2811:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:28 · section reference included

Turn strange behavior into an actionable issue

Production improvement begins with discovering failures you do not already know to test for. But a stream of reports that something weird happened does not establish what deserves attention. For each issue, Hylak wants two facts: when it started and how many users it affects.

Those facts support a practical investigation:

  1. Establish onset. If an issue began yesterday or several days ago, inspect what changed around that time.
  2. Identify candidate changes. A model replacement or a downstream change gives the investigation a concrete starting point.
  3. Measure reach. Count affected users and express that count as a percentage of users, so an anecdote can be placed in context.

Hylak contrasts an issue affecting three users with one affecting 100,000 users to illustrate why reach changes prioritization. Agents can produce an effectively unbounded variety of problems; onset and reach help a team choose where to spend its effort.

The right evaluation method also depends on the deployment’s scale. Hylak describes Raindrop customers ranging from five users to millions. A small internal enterprise application can still provide critical information, so a small audience does not imply low stakes.

At illustrative volumes of 10, 20 or 100 million messages per day, Hylak recommends experiments on a very small sample of free-tier traffic. For products with only five or ten users, he advises against experiments or A/B tests. These are different operating conditions, not different levels of commitment to quality: the available traffic changes which methods can produce useful evidence.

13:0813:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:08 · section reference included

A cluster is not an operational issue

The first of Hylak’s three tactical lessons from Raindrop is that clustering traces does not, by itself, define useful issues. Clustering can help with one-off exploration: a mass of logs becomes a set of patterns worth inspecting. He connects this exploratory work to Hamel Husain’s error analysis, but the useful activity is understanding failures, not merely obtaining clusters.

Ongoing operations need something more stable. As with ordinary telemetry, a team needs to know when an issue began and how much it has grown. Hylak points to the difficulty of tracking clusters reliably over time, including through temporal clustering. Cluster boundaries also need not match the product’s own definition of an issue. Two companies may reasonably group the same behaviors differently because their products and responsibilities differ.

Consider a wrong price quote and an incorrectly calculated refund. Both might fall into a cluster called price issues, but that category can hide separate root causes.

Observed behaviorBroad clusterUseful distinction to preserve
Wrong price quotedPrice issuesQuote failure
Wrong refund calculatedPrice issuesRefund calculation failure

A common subject does not establish a common cause. If the grouping erases that distinction, it becomes harder to tell which failure started recently, which is growing, and which fix should address it.

15:3115:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:31 · section reference included

Express trace classifiers as code

The second lesson applies code mode, familiar from discussions of MCP, to production traces. Instead of treating every classification as an open-ended reasoning task, write classifiers that can run in a sandbox. Hylak reports that this approach scales to production use. The slide gives concrete categories such as repeated tool calls, missed extraction and tool failures.

Slide showing a classifier code example checking repeated tool calls, missed extraction, and tool failures.Open full source frame
Many classifiers can be written as code.

For example, a TypeScript classifier can select explicit tool failures while preserving the trace and call identifiers needed for investigation. In this small example, quote_price succeeds and calculate_refund fails; the returned finding points to the refund call rather than grouping the entire trace under price issues.

typescript

type ToolCall = {
  id: string;
  name: string;
  status: "success" | "error";
};

type Trace = {
  id: string;
  toolCalls: ToolCall[];
};

type Finding = {
  traceId: string;
  callId: string;
  toolName: string;
  category: "tool-failure";
};

function classifyToolFailures(trace: Trace): Finding[] {
  return trace.toolCalls
    .filter(call => call.status === "error")
    .map(call => ({
      traceId: trace.id,
      callId: call.id,
      toolName: call.name,
      category: "tool-failure" as const,
    }));
}

const trace: Trace = {
  id: "trace-1",
  toolCalls: [
    { id: "call-1", name: "quote_price", status: "success" },
    { id: "call-2", name: "calculate_refund", status: "error" },
  ],
};

const findings = classifyToolFailures(trace);

The classifier detects an explicit error status; it does not establish whether a successful tool call returned a correct price or refund. That requires a different check. Hylak’s recommendation is to execute such classifiers in a sandbox; the function above supplies the classification logic, not the sandbox.

Raindrop has a feature to assist this workflow, but Hylak emphasizes that teams can implement it themselves. The operational idea is to make a useful distinction executable and apply it across traces.

18:1118:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:11 · section reference included

Detect a signal, then ask an agent to investigate

The final lesson separates anomaly detection from anomaly investigation. In Hylak’s experience, agents are poor at the former. Rather than asking an agent to search a large body of activity for something anomalous, give it an anomaly that has already been detected.

Extract deterministic signals wherever possible. Keyword frequency is his example: count occurrences, identify a spike, and make that spike the target of an agent’s investigation. A spike does not prove there is a product issue. It supplies a tangible change the agent can examine, explain and connect to the surrounding traces. The agent’s task becomes investigating a specific signal instead of deciding, from an undifferentiated mass of activity, what deserves attention.

18:3818:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:38 · section reference included

Resources

From the talk

  • Hamel Husain explains skills for reviewing traces, categorizing failures and building product-specific evaluations.

Transcript download
Full transcript

Loading transcript…