AI Engineer Summit 2025
Stop Guessing: Build Robust AI with Layered CoT
Loading slides…
Read the talk
Stop Guessing: Build Robust AI with Layered CoT
Specialized agents can divide a complex task, but their intermediate outputs still need checking. Layered CoT adds verification before each reasoning step becomes the basis for the next.
From a talk by Manish Sanwal
Divide the task among specialized agents
Consider a self-driving car: one agent detects pedestrians, another reads traffic signals, and a third selects a route. Each handles a distinct part of a larger task that requires coordination. This is Manish Sanwal’s opening illustration of a multi-agent system: a collection of specialized agents working together rather than one monolithic system handling everything.
The accompanying diagram places agents alongside orchestration and human oversight, connected to prompt context, an LLM, and tools. Specialization gives each agent a narrower task to tune. Modularity also lets developers update one agent without replacing the whole system. Sanwal argues that this supports flexibility and scalability, and that other agents can sometimes compensate when one encounters a problem; compensation depends on how the system is coordinated.
Dividing the work addresses who performs each task. Adding Chain-of-Thought, or CoT, addresses how an agent explains its output: it produces intermediate steps rather than only a conclusion. Those steps create places to inspect the work—and, eventually, to verify it before another step depends on it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make intermediate steps available for inspection
A detailed prompt can supply extensive context while still requesting only a final answer. The resulting output does not necessarily explain how the model reached its conclusion. CoT prompting instead asks for intermediate reasoning steps. The slide makes the distinction concrete: one path connects Input directly to Output; the other passes through three Thought boxes.
Breaking a complex problem into manageable steps gives a reader more to inspect. If an intermediate output contains a mistake, a developer can adjust the prompt or the surrounding process. That is an opportunity for debugging, not yet an automatic correction mechanism: someone or something still has to detect the mistake and intervene.
An inspectable explanation is not a guaranteed account of the model’s internal computation. Sanwal describes CoT as making reasoning visible and verifiable; the useful engineering interpretation is that generated intermediate outputs can be checked. Research on unfaithful explanations in Chain-of-Thought prompting shows why that distinction matters: explanations can omit influences that affected the answer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Where an unchecked chain can fail
Making intermediate steps visible leaves several failure modes unresolved:
- Prompt sensitivity. Small changes in wording or context can produce substantially different reasoning sequences, complicating reproducibility.
- Missing feedback. Ordinary stepwise generation does not itself include a mechanism that checks and corrects each step as it appears.
- Error propagation. A flawed early inference can become an assumption for later steps. Without ongoing checks, correction waits until inference has finished.
- Missed dependencies. A problem with multiple interdependent factors can be oversimplified if the sequence omits variables or fails to connect them.
The common weakness is that producing a sequence does not establish that its intermediate claims are valid.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate, verify, then advance
Layered CoT inserts verification between generating a step and allowing subsequent reasoning to depend on it. The initial output is a hypothesis to check, rather than an established premise. Sanwal describes two recurring stages:
- Generate a candidate step. The agent produces an initial piece of reasoning from the input prompt.
- Verify the candidate. Before proceeding, cross-reference that output against a structured knowledge base or an external database.
The suggested verification methods include a fact-checking algorithm, a contextual consistency check, or an ensemble of models assessing accuracy. These are alternative ways to perform the checking stage, not a specified implementation that combines all three.
Once a step is verified, the system advances to the next step and repeats the process. The intended constraint is that only accepted information influences later reasoning. This changes the dependency between steps:
| Approach | What happens before the next step? |
|---|---|
| Ordinary CoT | The model generates the preceding step. |
| Layered CoT | The preceding step is generated and checked. |
The consequential change is the verification boundary, not merely the presence of a knowledge base. The talk leaves acceptance thresholds, correction procedures, retry limits, and termination behavior unspecified; those decisions determine what happens when a candidate fails verification.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What the verification boundary is meant to improve
Checking each step creates an earlier opportunity for self-correction: a detected mistake can be addressed before it propagates through the chain. Sanwal also proposes that independent checks reduce sensitivity to prompt wording. Different prompts may still generate different candidates, but verification is intended to constrain which candidates become premises for later reasoning.
Discrete verification points can also make a system easier to audit. Instead of inspecting only a final answer, a reviewer can examine the intermediate outputs on which it depends and the checks applied to them. The intended result is a conclusion built from validated information, with a more legible path through the task.
These are proposed benefits, not measured guarantees in the recording. The companion paper’s performance discussion does not provide enough detail about tasks, model configurations, metric definitions, or evaluation procedures to establish a general accuracy or reproducibility gain. The strength of the approach therefore depends on the quality and coverage of its verification, rather than on adding a checking stage alone.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build verification into the system
Layered CoT returns to the modular system introduced at the start: specialized agents contribute to a shared task, with verification added to the reasoning workflow. Sanwal presents existing LLM tools as sufficient to implement the approach. The companion paper qualifies that feasibility with additional model interactions, external-data dependencies, and coordination overhead between agents. Integration is a system-design task, not a demonstrated turnkey feature.
The closing emphasis is on validating each inference before proceeding and making systems structured, explainable, and reliable alongside improving model capability. The conclusion slide groups the intended outcomes as verification at each step, self-correction, reduced prompt sensitivity, and repeatability and reproducibility. Those outcomes follow the same proposed design choice: give intermediate outputs an explicit check before allowing the system to build on them.
Sanwal closes by pointing to his Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems arXiv preprint. Its research direction extends the talk’s central engineering concern: trustworthy AI requires collaboration and validation within the system, with opportunities to inspect and correct its work as that work develops.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Sanwal's framework for checking intermediate reasoning against external sources and user feedback, with examples across three domains.
Further reading
Foundational research on using intermediate-reasoning examples in prompts for arithmetic, commonsense and symbolic tasks.
Experiments showing that plausible reasoning explanations can conceal the influence of biases in model inputs.
Full transcript
Loading transcript…