AI Engineer World's Fair 2026
Don't Let the LLM Drive - Ornella Bahidika & Joel Allou, Microsoft
Loading slides…
Read the talk
Don't Let the LLM Drive
Ace’s live voice tutor puts lesson progression in a state machine, leaving the model to perform bounded actions while a harness validates results and decides what happens next.
From a talk by Ornella Bahidika and Joel Allou
Before you start: Basic familiarity with LLM applications and application state is helpful; the TypeScript example illustrates who owns a lesson transition.
The lesson ends halfway through
A multi-step agent completes its demo perfectly. Then a real user arrives, and halfway through the interaction it declares itself finished, skips a state, or loops. For Ace, the live AI voice tutor built by Ornella Bahidika and Joel Allou, that failure would mean losing the structure of a lesson that needs to run from introduction to completion.
The tempting repair is to add more instructions: remember the steps, do not skip ahead, do not stop early. Bahidika frames the underlying issue as control flow. The model can deliver a good explanation without reliably tracking whether it is on step three of six. Her analogy separates the talent from the director: the model delivers the line; the surrounding harness keeps the production moving through the right sequence.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give each step a narrow contract
Ace organizes a lesson as a small state machine: intro → teach → check → grade → advance → wrap. Each step gives the model one bounded assignment and asks for its result. The harness then validates that result, advances the state, and selects the next assignment. The model does not determine the current lesson state.
A minimal TypeScript sketch of that boundary can make the ownership explicit. Here, a teach operation returns an explanation; only application code can move the lesson to check.
typescript
type Lesson = {
state: "teach" | "check";
topic: string;
explanation: string | null;
};
type TeachInput = {
action: "teach";
topic: string;
};
async function teach(
lesson: Lesson,
callModel: (input: TeachInput) => Promise<unknown>
): Promise<Lesson> {
if (lesson.state !== "teach") {
throw new Error("Lesson is not ready to teach");
}
const result = await callModel({
action: "teach",
topic: lesson.topic,
});
if (typeof result !== "string" || result.trim() === "") {
throw new Error("Expected a nonempty explanation");
}
return {
...lesson,
explanation: result.trim(),
state: "check",
};
}
The model supplies content, not a nextState field. An invalid response prevents this transition. The string check illustrates the validation boundary; Ace’s actual content-validation rules are not detailed in the talk.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Smaller assignments enable a smaller model
Allou starts with Claude Opus 4.7 as an example of a frontier model people might use for thinking, processing, and everything between. A live tutor has a more specific constraint: it must speak back and forth with students while remaining reliable, affordable, and fast. Handing the entire lesson to a powerful model leaves that model responsible for coordination as well as conversation.
The harness narrows that responsibility by answering three questions before each action:
- What step is active now?
- Which steps are allowed to follow it?
- What concrete input does the model need to perform only the current action?
This confines each model call to the work required at that moment, rather than asking it to reconstruct the lesson’s control logic.
Allou reports that Claude Haiku 4.5, despite having less reasoning capability than the larger model, meets Ace’s expected performance with this harness. He reports savings in money, time, and latency, but supplies no measurements or controlled tutoring comparison. The demonstration then turns to a recorded lesson, with harness logs displayed on the right side.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Control the operations around the conversation
The narrated recording shows that the harness covers more than the order of spoken exchanges. Allou identifies several operational responsibilities:
- Section instructions: Give the model the specific material to speak about and the action to perform.
- Whiteboard drawing: Handle the drawing actions that accompany the lesson.
- Queue clearing: Include explicit steps for clearing the queue.
- Lesson completion: Define how the lesson ends instead of leaving completion to an improvised model decision.
The completed screen offers Practice: Grammar Foundations and Back to Home, with the developer console alongside it. The individual operations come from Allou’s narration; the console text in this frame is too small to read in detail.
New scenarios become additions to the state machine and lesson logic. The model’s local job remains the same: receive an action-specific input and return that action’s output. Reliability therefore depends on the harness covering the situations the lesson can encounter, including operational steps that are easy to overlook when focusing only on the conversation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The model proposes; the harness decides
That division becomes consequential at three decision points in Ace:
| Decision | Harness responsibility |
|---|---|
| Is the lesson done? | Own the completion decision. |
| Did the student get it right? | Own the decision about correctness and intended learning. |
| What comes next? | Select the next step. |
A model output can be a proposal without becoming an authoritative state change. These decisions determine whether the tutor continues teaching, evaluates progress, or ends the lesson.
Allou says the questions and actions within these categories are engineered outside the model. His description of the model as not having to think concerns this bounded responsibility: it receives an input and produces an output, while the application owns the surrounding decisions. The talk does not specify the grading rules or how Ace establishes that a student has learned the material, so the architectural boundary is clearer than the assessment implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move unreliable control flow into code
The same division can apply to coding agents, applications, runbooks, and onboarding flows. Allou’s diagnostic is qualitative: if reliable completion feels like a coin flip, examine how many workflow decisions the model owns. Move those decisions into the surrounding application and give the model simpler inputs for bounded actions.
The ending returns to the voice tutor with a useful self-correction: let the model talk, but do not let it drive. A model can explain, converse, and propose useful outputs while the harness remains responsible for where the lesson is, whether a result is acceptable, and what happens next.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Anthropic's release announcement for the larger model used as the talk's comparison point.
Release announcement covering Haiku 4.5's capabilities, launch pricing, and evaluation methodology.
Further reading
- Building effective agentsArticle
Practical workflow patterns, including predefined control paths and programmatic checks between model calls.
Full transcript
Loading transcript…