AI Engineer Summit 2025

Stop Guessing: Build Robust AI with Layered CoT

Loading slides…

Read the talk

Stop Guessing: Build Robust AI with Layered CoT

Specialized agents can divide a complex task, but their intermediate outputs still need checking. Layered CoT adds verification before each reasoning step becomes the basis for the next.

From a talk by Manish Sanwal

Divide the task among specialized agents

Consider a self-driving car: one agent detects pedestrians, another reads traffic signals, and a third selects a route. Each handles a distinct part of a larger task that requires coordination. This is Manish Sanwal’s opening illustration of a multi-agent system: a collection of specialized agents working together rather than one monolithic system handling everything.

The accompanying diagram places agents alongside orchestration and human oversight, connected to prompt context, an LLM, and tools. Specialization gives each agent a narrower task to tune. Modularity also lets developers update one agent without replacing the whole system. Sanwal argues that this supports flexibility and scalability, and that other agents can sometimes compensate when one encounters a problem; compensation depends on how the system is coordinated.

Diagram showing agent orchestration beside a human-in-the-loop symbol and stacked agents, with arrows toward prompt context, an LLM, and tools.Open full source frame
Multi-agent system with orchestration, human oversight, prompt context, an LLM, and tools.

Dividing the work addresses who performs each task. Adding Chain-of-Thought, or CoT, addresses how an agent explains its output: it produces intermediate steps rather than only a conclusion. Those steps create places to inspect the work—and, eventually, to verify it before another step depends on it.

0:330:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:33 · section reference included

Make intermediate steps available for inspection

A detailed prompt can supply extensive context while still requesting only a final answer. The resulting output does not necessarily explain how the model reached its conclusion. CoT prompting instead asks for intermediate reasoning steps. The slide makes the distinction concrete: one path connects Input directly to Output; the other passes through three Thought boxes.

Top row connects Input directly to Output. Bottom row connects Input through three Thought boxes to Output.Open full source frame
Standard prompting and chain-of-thought prompting shown as two paths from input to output.

Breaking a complex problem into manageable steps gives a reader more to inspect. If an intermediate output contains a mistake, a developer can adjust the prompt or the surrounding process. That is an opportunity for debugging, not yet an automatic correction mechanism: someone or something still has to detect the mistake and intervene.

An inspectable explanation is not a guaranteed account of the model’s internal computation. Sanwal describes CoT as making reasoning visible and verifiable; the useful engineering interpretation is that generated intermediate outputs can be checked. Research on unfaithful explanations in Chain-of-Thought prompting shows why that distinction matters: explanations can omit influences that affected the answer.

2:222:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:22 · section reference included

Where an unchecked chain can fail

Making intermediate steps visible leaves several failure modes unresolved:

  • Prompt sensitivity. Small changes in wording or context can produce substantially different reasoning sequences, complicating reproducibility.
  • Missing feedback. Ordinary stepwise generation does not itself include a mechanism that checks and corrects each step as it appears.
  • Error propagation. A flawed early inference can become an assumption for later steps. Without ongoing checks, correction waits until inference has finished.
  • Missed dependencies. A problem with multiple interdependent factors can be oversimplified if the sequence omits variables or fails to connect them.

The common weakness is that producing a sequence does not establish that its intermediate claims are valid.

4:144:24
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:14 · section reference included

Generate, verify, then advance

Layered CoT inserts verification between generating a step and allowing subsequent reasoning to depend on it. The initial output is a hypothesis to check, rather than an established premise. Sanwal describes two recurring stages:

  1. Generate a candidate step. The agent produces an initial piece of reasoning from the input prompt.
  2. Verify the candidate. Before proceeding, cross-reference that output against a structured knowledge base or an external database.

The suggested verification methods include a fact-checking algorithm, a contextual consistency check, or an ensemble of models assessing accuracy. These are alternative ways to perform the checking stage, not a specified implementation that combines all three.

Stacked agents connect to prompt context and tools, with two Thought boxes leading toward an LLM. A Knowledge base cylinder and curved arrows appear above the Thought boxes.Open full source frame
A knowledge base sits above the thought sequence in a multi-agent workflow.

Once a step is verified, the system advances to the next step and repeats the process. The intended constraint is that only accepted information influences later reasoning. This changes the dependency between steps:

ApproachWhat happens before the next step?
Ordinary CoTThe model generates the preceding step.
Layered CoTThe preceding step is generated and checked.

The consequential change is the verification boundary, not merely the presence of a knowledge base. The talk leaves acceptance thresholds, correction procedures, retry limits, and termination behavior unspecified; those decisions determine what happens when a candidate fails verification.

6:066:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:06 · section reference included

What the verification boundary is meant to improve

Checking each step creates an earlier opportunity for self-correction: a detected mistake can be addressed before it propagates through the chain. Sanwal also proposes that independent checks reduce sensitivity to prompt wording. Different prompts may still generate different candidates, but verification is intended to constrain which candidates become premises for later reasoning.

Discrete verification points can also make a system easier to audit. Instead of inspecting only a final answer, a reviewer can examine the intermediate outputs on which it depends and the checks applied to them. The intended result is a conclusion built from validated information, with a more legible path through the task.

These are proposed benefits, not measured guarantees in the recording. The companion paper’s performance discussion does not provide enough detail about tasks, model configurations, metric definitions, or evaluation procedures to establish a general accuracy or reproducibility gain. The strength of the approach therefore depends on the quality and coverage of its verification, rather than on adding a checking stage alone.

7:488:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:48 · section reference included

Build verification into the system

Layered CoT returns to the modular system introduced at the start: specialized agents contribute to a shared task, with verification added to the reasoning workflow. Sanwal presents existing LLM tools as sufficient to implement the approach. The companion paper qualifies that feasibility with additional model interactions, external-data dependencies, and coordination overhead between agents. Integration is a system-design task, not a demonstrated turnkey feature.

The closing emphasis is on validating each inference before proceeding and making systems structured, explainable, and reliable alongside improving model capability. The conclusion slide groups the intended outcomes as verification at each step, self-correction, reduced prompt sensitivity, and repeatability and reproducibility. Those outcomes follow the same proposed design choice: give intermediate outputs an explicit check before allowing the system to build on them.

Conclusion slide with four bullets: Agent verifies responses at each step; Self correcting property; Not sensitive to prompt designs; Repeatability and reproducibility. Presenter appears at right.Open full source frame
The conclusion lists verification, self-correction, prompt sensitivity, and reproducibility claims.

Sanwal closes by pointing to his Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems arXiv preprint. Its research direction extends the talk’s central engineering concern: trustworthy AI requires collaboration and validation within the system, with opportunities to inspect and correct its work as that work develops.

9:109:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:10 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…