AI Engineer Summit 2025

Building enterprise LLM agents that work

Loading slides…

Read the talk

Building enterprise LLM agents that work

Reliable enterprise agents require clear tool contracts, constrained orchestration, explicit human review, and evaluations that expose where a tool-using workflow fails.

From a talk by Shaan Desai

Before you start: Familiarity with LLM tool calling and retrieval-augmented generation will help; no particular agent framework is required.

How do the pieces become a reliable agent?

How do you turn a model, a collection of tools, and an orchestration framework into an agent that operates safely at enterprise scale? Customer support assistants, personal assistants, retrieval-augmented generation (RAG) agents, and financial analysis agents all face this integration problem. Each component may work individually while the complete workflow remains difficult to scale, secure, and operate smoothly.

The difficulty starts with the number of choices: frameworks, tools, models, orchestration strategies, and evaluation criteria must fit together in one pipeline. Shaan Desai’s experience at Cohere suggests a practical decision sequence: choose a framework you can debug, decide how much agent coordination the task needs, and establish how you will evaluate the resulting behavior.

Introduction slide listing customer support, personal assistants, reasoning agents on RAG, and financial analysis, followed by enterprise implementation challenges and talk topics.Open full source frame
Enterprise agent use cases and the challenge of building scalable, safe systems.
0:380:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:38 · section reference included

Choose a framework you can debug

Frameworks such as AutoGen, CrewAI, and LangChain offer different tradeoffs. Three criteria make those tradeoffs concrete:

CriterionPractical question
ObservabilityCan you inspect, debug, and fix failures?
Setup costHow quickly can you assemble the agent and iterate?
SupportAre the documentation, models, tools, and required features available?

These criteria matter together: a quick initial setup is less valuable if resolving the first difficult failure becomes expensive.

For large enterprise agents with demanding observability requirements, Desai recommends a native implementation or LangGraph. For quick tests and proofs of concept, he recommends CrewAI and AutoGen because they reduce setup work and make it easy to combine prebuilt agents and tools. These are recommendations from the talk’s period, not a permanent ranking; AutoGen’s repository now carries a maintenance-mode notice.

Frameworks slide recommending native development or LangGraph for enterprise work, and CrewAI or AutoGen for quick tests, with supporting bullets.Open full source frame
Framework recommendations for enterprise applications and quick prototypes.

Cohere’s approach is to keep improving integrations across frameworks as their capabilities evolve. The useful outcome is a spectrum of options suited to different applications, rather than one framework winning every use case. Once the framework is selected, the next decision is how to organize the agent itself.

1:592:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:59 · section reference included

Start with one agent and a clear tool contract

Single-agent execution, multi-agent coordination, and human feedback are explicit design choices. Start with a single LLM and a handful of tools. Before adding more agents, make the tools easier for that model to use. Desai identifies careful tool specifications as a substantial source of improvement.

One Cohere client had a long list of APIs whose specifications accepted up to 10–15 parameters. The request was to get a model to call those APIs successfully. The response was to simplify the interface presented to the model: write clear descriptions, provide precise examples of valid calls, and replace complex nested dictionaries with simpler inputs such as list, str, or float where appropriate. The parameter count describes that client’s problem; it is not a universal limit on reliable tool use.

Apply the same discipline to instructions. A short, direct list makes the required behavior easier to identify. Desai reports that long instruction sets can instead confuse the model and induce hallucinations. Adding more instructions is therefore not automatically a better specification.

5:005:15
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:00 · section reference included

Retrieve the history that matters

Conversation history creates a related problem. Desai reports hallucinations in long conversations across multiple models and frameworks, using conversations over 20 turns as an example rather than a universal cutoff. His recommendation is to cache the history and retrieve relevant portions when a new query needs them. The history remains available, but the model does not need every previous exchange in every request. Whether that improves a particular agent must be assessed through the evaluation process that follows.

6:497:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:49 · section reference included

Constrain routing before expanding autonomy

When the application does need multiple agents, the architecture becomes a collection of simpler agents coordinated by a router. AutoGen is one framework Desai cites for this orchestration. Robustness depends on three things working together: a capable routing model, good reasoning, and sub-agents with constrained responsibilities. Adding agents does not remove the need to specify their behavior.

The router needs clear tool descriptions and explicit routing instructions, including edge cases. Specify what happens when work moves from the router to a sub-agent and then onward to another agent. Those handoff instructions help the model choose the next stage instead of repeatedly attempting actions that do not advance the task.

Each sub-agent should own a specific, independent task and have a small set of tools with which to complete it. This preserves the simplicity of the single-agent design inside the larger system: coordination grows, but each participant’s responsibility remains narrow.

7:367:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:36 · section reference included

Make human review an execution rule

An agent’s ability to act autonomously creates a safety boundary. Consider a Gmail agent: before sending an email, it should ask the user to approve the proposed message. At that point, the email is still pending; proposing a send is not permission to execute it. Similar review requirements arise in HR support and financial analysis.

Human review can be triggered by codified rules. Rather than leaving intervention to chance, define the conditions under which the workflow must stop for a person.

Human-in-the-loop slide with permission and user-input examples beside a branching workflow diagram containing a person icon.Open full source frame
Human review examples for email, HR support, and financial agents.

Those rules can operate on either side of a tool call:

  • Before execution: request permission for a proposed action, such as sending an email.
  • After execution: review returned information before allowing the agent to process it further.

The second checkpoint serves a different purpose: it governs the use of a result, rather than authorizing an action that has already happened.

9:219:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:21 · section reference included

Evaluate the path to the answer

A successful agent must choose the right tool at the right time, pass accurate parameters, reason over executed results, and recover when something goes wrong. The user primarily cares about the final answer. The developer also needs to see the intermediate decisions that produced it, because a plausible answer alone does not reveal whether those decisions were correct.

The weather example makes those stages concrete. A user asks for the weather in New York City on February fifth. The intended workflow is:

  1. Select the appropriate weather tool.
  2. Pass the location and date as the required parameters.
  3. Receive the tool’s executed result.
  4. Interpret that result and produce the final response.

The example response says New York City will be mostly sunny; it illustrates the workflow, not a verified forecast. Each stage gives the developer a separate place to inspect a failure.

Cohere builds golden evaluation sets that record expectations across this entire path:

Evaluation itemWhat it establishes
User queryThe task presented to the agent
Expected function callWhich tool should be selected
Expected parametersWhat inputs the tool should receive
Expected tool outputWhat result the agent should reason over
Expected final responseWhat the user should receive

Running this corpus through the agent framework helps locate the stage where behavior diverges from expectations. A tool-selection failure calls for a different investigation from incorrect parameters or a final answer that misinterprets a correct tool result. This is why observability belongs in framework selection, not just in the debugging work added afterward.

11:0211:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:02 · section reference included

Match mitigation to the observed failure

Autonomous agents do fail, so evaluation must lead to a repair strategy. For failures with low severity or low frequency, Desai recommends starting with prompt engineering: improve the tool API specifications and clarify the inputs. These changes can address remaining performance gaps without immediately changing the model.

For task-specific tool failures or hallucinations in the 10–20% range, Desai recommends building a targeted annotation dataset. This is a practical heuristic from Cohere’s experience, not a universal decision boundary. The talk does not specify how that dataset is subsequently used, so this step should not automatically be equated with fine-tuning.

For high failure rates, Cohere moves to a larger corpus built with synthetic data and fine-tunes the model. Desai particularly associates this approach with APIs that are difficult to call or have similar names that the model must disambiguate. The escalation runs from clarifying the existing contract to supplying more task-specific training material when the underlying difficulty persists.

13:3213:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:32 · section reference included

Improve the model’s tool-calling ability

Framework design, orchestration, and failure mitigation sit alongside another line of work: improving the base model’s ability to call tools. Desai points to BFCL V3 as a benchmark for tool calling and describes Cohere’s performance there favorably. The claim remains qualitative here; it does not establish a numerical score or ranking.

Desai highlights a 7-billion-parameter model as part of the demand for lightweight tool-calling models. The model is unnamed in his spoken explanation. Its role in the architecture is clear, however: stronger tool calling at the model level complements the constraints and evaluations built around it.

15:0715:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:07 · section reference included

North brings the workflow into one product

North packages the preceding lessons from frameworks, agent design, and enterprise deployment. Desai describes the version presented in the talk as a single-container deployment with RAG, access to vector databases and search, and connections to applications including Gmail, Outlook, Drive, and Slack. The package is intended for both using and building agentic applications.

The closing demo connects North to Gmail, Slack, Salesforce, and Google Drive. A question about Salesforce opportunities initiates retrieval of a relevant document. The interface exposes what Desai calls a reasoning chain, along with the tools called and their outputs. The visible activity details sit above a tabular result, making the work behind the response available for inspection.

Product Overview: North slide containing an assistant conversation with several activity rows expanded above a results table, alongside the presenter.Open full source frame
North demo showing expanded activity details above a tabular result.

The next lookup concerns recent conversations. Desai describes Salesforce calls using SQL-like queries, then shows how a user can correct the tool choice. The correction leads the system to revise its approach and use Gmail to retrieve the relevant information. The substantive end of the demo is this repair loop: inspect the agent’s activity, identify an unsuitable tool selection, and redirect it toward the source the task actually requires.

15:5816:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:58 · section reference included

Resources

From the talk

  • Explains multi-turn and multi-step tool-calling tasks and how execution states and required calls are evaluated.

  • Cohere's announcement of a workspace combining language models, search, and automation.

  • Stateful agent orchestration with persistence, human oversight, and debugging integrations.

  • Multi-agent application framework with examples and documentation; the repository now carries a maintenance-mode notice.

  • Python framework for collaborating agents and explicitly controlled workflows, with installation and example projects.

  • Official source repository and entry point to LangChain's agent development ecosystem.

Transcript download
Full transcript

Loading transcript…