AI Engineer Summit 2025
Building enterprise LLM agents that work
Loading slides…
Read the talk
Building enterprise LLM agents that work
Reliable enterprise agents require clear tool contracts, constrained orchestration, explicit human review, and evaluations that expose where a tool-using workflow fails.
From a talk by Shaan Desai
Before you start: Familiarity with LLM tool calling and retrieval-augmented generation will help; no particular agent framework is required.
How do the pieces become a reliable agent?
How do you turn a model, a collection of tools, and an orchestration framework into an agent that operates safely at enterprise scale? Customer support assistants, personal assistants, retrieval-augmented generation (RAG) agents, and financial analysis agents all face this integration problem. Each component may work individually while the complete workflow remains difficult to scale, secure, and operate smoothly.
The difficulty starts with the number of choices: frameworks, tools, models, orchestration strategies, and evaluation criteria must fit together in one pipeline. Shaan Desai’s experience at Cohere suggests a practical decision sequence: choose a framework you can debug, decide how much agent coordination the task needs, and establish how you will evaluate the resulting behavior.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Choose a framework you can debug
Frameworks such as AutoGen, CrewAI, and LangChain offer different tradeoffs. Three criteria make those tradeoffs concrete:
| Criterion | Practical question |
|---|---|
| Observability | Can you inspect, debug, and fix failures? |
| Setup cost | How quickly can you assemble the agent and iterate? |
| Support | Are the documentation, models, tools, and required features available? |
These criteria matter together: a quick initial setup is less valuable if resolving the first difficult failure becomes expensive.
For large enterprise agents with demanding observability requirements, Desai recommends a native implementation or LangGraph. For quick tests and proofs of concept, he recommends CrewAI and AutoGen because they reduce setup work and make it easy to combine prebuilt agents and tools. These are recommendations from the talk’s period, not a permanent ranking; AutoGen’s repository now carries a maintenance-mode notice.
Cohere’s approach is to keep improving integrations across frameworks as their capabilities evolve. The useful outcome is a spectrum of options suited to different applications, rather than one framework winning every use case. Once the framework is selected, the next decision is how to organize the agent itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with one agent and a clear tool contract
Single-agent execution, multi-agent coordination, and human feedback are explicit design choices. Start with a single LLM and a handful of tools. Before adding more agents, make the tools easier for that model to use. Desai identifies careful tool specifications as a substantial source of improvement.
One Cohere client had a long list of APIs whose specifications accepted up to 10–15 parameters. The request was to get a model to call those APIs successfully. The response was to simplify the interface presented to the model: write clear descriptions, provide precise examples of valid calls, and replace complex nested dictionaries with simpler inputs such as list, str, or float where appropriate. The parameter count describes that client’s problem; it is not a universal limit on reliable tool use.
Apply the same discipline to instructions. A short, direct list makes the required behavior easier to identify. Desai reports that long instruction sets can instead confuse the model and induce hallucinations. Adding more instructions is therefore not automatically a better specification.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retrieve the history that matters
Conversation history creates a related problem. Desai reports hallucinations in long conversations across multiple models and frameworks, using conversations over 20 turns as an example rather than a universal cutoff. His recommendation is to cache the history and retrieve relevant portions when a new query needs them. The history remains available, but the model does not need every previous exchange in every request. Whether that improves a particular agent must be assessed through the evaluation process that follows.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Constrain routing before expanding autonomy
When the application does need multiple agents, the architecture becomes a collection of simpler agents coordinated by a router. AutoGen is one framework Desai cites for this orchestration. Robustness depends on three things working together: a capable routing model, good reasoning, and sub-agents with constrained responsibilities. Adding agents does not remove the need to specify their behavior.
The router needs clear tool descriptions and explicit routing instructions, including edge cases. Specify what happens when work moves from the router to a sub-agent and then onward to another agent. Those handoff instructions help the model choose the next stage instead of repeatedly attempting actions that do not advance the task.
Each sub-agent should own a specific, independent task and have a small set of tools with which to complete it. This preserves the simplicity of the single-agent design inside the larger system: coordination grows, but each participant’s responsibility remains narrow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make human review an execution rule
An agent’s ability to act autonomously creates a safety boundary. Consider a Gmail agent: before sending an email, it should ask the user to approve the proposed message. At that point, the email is still pending; proposing a send is not permission to execute it. Similar review requirements arise in HR support and financial analysis.
Human review can be triggered by codified rules. Rather than leaving intervention to chance, define the conditions under which the workflow must stop for a person.
Those rules can operate on either side of a tool call:
- Before execution: request permission for a proposed action, such as sending an email.
- After execution: review returned information before allowing the agent to process it further.
The second checkpoint serves a different purpose: it governs the use of a result, rather than authorizing an action that has already happened.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluate the path to the answer
A successful agent must choose the right tool at the right time, pass accurate parameters, reason over executed results, and recover when something goes wrong. The user primarily cares about the final answer. The developer also needs to see the intermediate decisions that produced it, because a plausible answer alone does not reveal whether those decisions were correct.
The weather example makes those stages concrete. A user asks for the weather in New York City on February fifth. The intended workflow is:
- Select the appropriate weather tool.
- Pass the location and date as the required parameters.
- Receive the tool’s executed result.
- Interpret that result and produce the final response.
The example response says New York City will be mostly sunny; it illustrates the workflow, not a verified forecast. Each stage gives the developer a separate place to inspect a failure.
Cohere builds golden evaluation sets that record expectations across this entire path:
| Evaluation item | What it establishes |
|---|---|
| User query | The task presented to the agent |
| Expected function call | Which tool should be selected |
| Expected parameters | What inputs the tool should receive |
| Expected tool output | What result the agent should reason over |
| Expected final response | What the user should receive |
Running this corpus through the agent framework helps locate the stage where behavior diverges from expectations. A tool-selection failure calls for a different investigation from incorrect parameters or a final answer that misinterprets a correct tool result. This is why observability belongs in framework selection, not just in the debugging work added afterward.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Match mitigation to the observed failure
Autonomous agents do fail, so evaluation must lead to a repair strategy. For failures with low severity or low frequency, Desai recommends starting with prompt engineering: improve the tool API specifications and clarify the inputs. These changes can address remaining performance gaps without immediately changing the model.
For task-specific tool failures or hallucinations in the 10–20% range, Desai recommends building a targeted annotation dataset. This is a practical heuristic from Cohere’s experience, not a universal decision boundary. The talk does not specify how that dataset is subsequently used, so this step should not automatically be equated with fine-tuning.
For high failure rates, Cohere moves to a larger corpus built with synthetic data and fine-tunes the model. Desai particularly associates this approach with APIs that are difficult to call or have similar names that the model must disambiguate. The escalation runs from clarifying the existing contract to supplying more task-specific training material when the underlying difficulty persists.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Improve the model’s tool-calling ability
Framework design, orchestration, and failure mitigation sit alongside another line of work: improving the base model’s ability to call tools. Desai points to BFCL V3 as a benchmark for tool calling and describes Cohere’s performance there favorably. The claim remains qualitative here; it does not establish a numerical score or ranking.
Desai highlights a 7-billion-parameter model as part of the demand for lightweight tool-calling models. The model is unnamed in his spoken explanation. Its role in the architecture is clear, however: stronger tool calling at the model level complements the constraints and evaluations built around it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
North brings the workflow into one product
North packages the preceding lessons from frameworks, agent design, and enterprise deployment. Desai describes the version presented in the talk as a single-container deployment with RAG, access to vector databases and search, and connections to applications including Gmail, Outlook, Drive, and Slack. The package is intended for both using and building agentic applications.
The closing demo connects North to Gmail, Slack, Salesforce, and Google Drive. A question about Salesforce opportunities initiates retrieval of a relevant document. The interface exposes what Desai calls a reasoning chain, along with the tools called and their outputs. The visible activity details sit above a tabular result, making the work behind the response available for inspection.
The next lookup concerns recent conversations. Desai describes Salesforce calls using SQL-like queries, then shows how a user can correct the tool choice. The correction leads the system to revise its approach and use Gmail to retrieve the relevant information. The substantive end of the demo is this repair loop: inspect the agent’s activity, identify an unsuitable tool selection, and redirect it toward the source the task actually requires.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Explains multi-turn and multi-step tool-calling tasks and how execution states and required calls are evaluated.
Cohere's announcement of a workspace combining language models, search, and automation.
Stateful agent orchestration with persistence, human oversight, and debugging integrations.
Multi-agent application framework with examples and documentation; the repository now carries a maintenance-mode notice.
Python framework for collaborating agents and explicitly controlled workflows, with installation and example projects.
Official source repository and entry point to LangChain's agent development ecosystem.
Further reading
Model specifications, usage examples, tool-use capabilities, and access and licensing conditions for Cohere's 7B model.
Walkthrough of tool calls, application execution, returned results, and response generation using Cohere.
Full transcript
Loading transcript…