AI Engineer World's Fair 2026

Your Voice Agent Doesn't Need a Frontier Model

Loading slides…

Read the talk

Your Voice Agent Doesn’t Need a Frontier Model

Ace keeps its voice tutor responsive by moving lesson state, mastery tracking, and next-step decisions into code, leaving a small model to turn those decisions into speech.

From a talk by Joel Allou and Ornella Bahidika

The pause is the problem

A student asks a voice tutor a question. The tutor goes silent. Is it thinking, or has the app stopped working? For Ornella Bahidika and Joel Allou, who built the live AI voice tutor Ace, that ambiguity is the opening design problem. Bahidika describes even a one-second pause as enough to make the interaction feel broken.

Slide reading “A few seconds pause feels broken,” with two waveform sections separated by a gap labeled “dead air · 3.0s.”Open full source frame
A pause in a voice conversation feels broken.

When an answer feels wrong, reaching for a smarter, larger model is an understandable response. But a voice application also has to preserve the rhythm of conversation. Ace’s target is to start talking in about 950 milliseconds. In that budget, a model that spends a full second thinking has already waited too long, however good its eventual answer. The target is Ace’s design constraint, rather than a universal threshold for conversation.

0:000:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Give the model a turn summary, not the whole lesson

Ace makes a small model viable by taking responsibilities away from it. The surrounding system tracks what happened in the lesson, what the student knows, and what should happen next. Every turn, it hands the model a summary. The model’s remaining job is to express the prepared result conversationally.

Allou uses Anthropic’s Claude Opus 4.7 as the frontier-model example, initially calling it Claude 4.7. Such a model can reason through a lesson, interpret a student’s question, and work out an answer. But doing that reasoning during the conversational turn can consume the seconds the voice interface cannot spare. Ace moves that work outside the speaking model.

A state machine encodes anticipated lesson scenarios and coordinates transitions between steps. An additional intelligent layer derives the student’s mastery and helps determine whether the lesson is complete. Together, these components decide the next action, the content to display, and how a question should be answered. Their output goes to the model to verbalize. This separation makes the speaking model a consumer of lesson decisions instead of the owner of lesson progression.

1:041:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:04 · section reference included

The same question, two different systems

The recorded comparison starts with Opus 4.7 without Ace’s implementation. A simple question leaves the model visibly thinking before returning an answer. The second example asks the same question using the smaller Haiku 4.5 with the surrounding architecture in place.

ConfigurationReported response
Opus 4.7 without Ace’s scaffoldingAllou describes a couple of seconds to return an answer.
Haiku 4.5 with Ace’s scaffoldingAllou reports an answer in about 900 milliseconds.

Both the model and the scaffolding change, so the comparison illustrates the combined design rather than isolating either contribution. The recording does not specify the timing boundaries or repeated-trial conditions; the reported response time should not automatically be read as time to first text token or first audible speech.

Allou attributes the improvement to moving logic and reasoning into code before the speaking model is called. He also describes the smaller model as cost effective, without supplying cost figures. The visible tutor result shows a lesson titled “The Eight Parts of Speech,” alongside chat messages and a developer panel. The interaction feels almost instant to him because the decisions needed for the response have already been made.

Slide headed “It feels instant” beside a tutor interface showing “The Eight Parts of Speech,” chat messages, and a developer panel.Open full source frame
The tutor demo displays “The Eight Parts of Speech.”
3:073:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:07 · section reference included

Pay for structure in code

The smaller model creates its own engineering obligation. Allou reports that Haiku 4.5 without scaffolding tends to drift over long structures and needs strict rules to stay organized. Scaffolding is the price of narrowing the model’s job. He frames that price as development work paid in code, rather than reasoning performed afresh on every conversational turn. His selection rule is to choose the fastest model the latency budget allows, then spend the remaining effort on scaffolding.

Dark slide with the model-selection rule in white and “Spend the rest on scaffolding” highlighted in orange.Open full source frame
Pick the fastest model your latency allows. Spend the rest on scaffolding.

That work means enumerating scenarios, defining transitions, and deciding how each situation should be handled. A small TypeScript teaching example can make the boundary concrete. Suppose a parts-of-speech lesson asks the student to identify a noun. An upstream evaluator supplies correct; the transition function updates lesson state and prepares the speaking instruction:

typescript

type LessonState = {
  step: "noun-practice" | "verb-practice";
  nounMastered: boolean;
};

type Evaluation = {
  answer: string;
  correct: boolean;
};

function handleNounAnswer(
  state: LessonState,
  evaluation: Evaluation,
) {
  if (state.step !== "noun-practice") {
    throw new Error("Expected noun practice");
  }

  const next: LessonState = evaluation.correct
    ? { step: "verb-practice", nounMastered: true }
    : { ...state };

  return {
    state: next,
    display: evaluation.correct ? "Verbs" : "Nouns",
    speakingSummary: {
      studentAnswer: evaluation.answer,
      correct: evaluation.correct,
      instruction: evaluation.correct
        ? "Confirm the noun answer, then introduce verbs."
        : "Explain that a noun names a person, place, or thing; ask again.",
    },
  };
}

const preparedTurn = handleNounAnswer(
  { step: "noun-practice", nounMastered: false },
  { answer: "cat", correct: true },
);

Here, the example’s deliberately simple mastery rule advances after one correct answer. Ace’s actual mastery algorithm is not specified. The architectural point is the handoff: preparedTurn contains the next state, display choice, and speaking summary before any language generation occurs. The model receives the prepared instruction; it does not choose the transition or assign mastery.

4:154:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:15 · section reference included

A small model inside a larger system

Allou extends the pattern beyond voice tutoring to other real-time applications where latency matters, and to high-volume workloads where repeated reasoning carries a recurring cost. The surrounding system owns the durable structure: state, scenarios, decisions, and progression. The model supplies the language at the point where language is needed. In that architecture, the model can become the smallest part of the system while the application remains capable of guiding a complete interaction.

5:165:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:16 · section reference included

Resources

From the talk

  • Anthropic's launch announcement for Opus 4.7, covering capabilities, availability, pricing, and reasoning-effort controls.

  • Anthropic's introduction to its small Haiku 4.5 model, including low-latency use cases and launch pricing.

Transcript download
Full transcript

Loading transcript…