AI Engineer World's Fair 2025
The State of MCP Observability: Observable.tools — Alex Volkov and Benjamin Eckel, Weights & Biases and Dylibso
Loading slides…
Read the talk
The State of MCP Observability: Observable.tools
MCP tools can hide the work behind an agent’s calls. OpenTelemetry context propagation connects those calls to server execution, while shared conventions make the resulting traces portable.
From a talk by Alex Volkov and Benjamin Eckel
Before you start: Familiarity with an AI agent calling tools through an MCP client and server is useful; the article introduces the tracing concepts it uses.
When adding tools makes an agent harder to observe
An agent works in production, but something goes wrong after it gains tools through MCP. Where did the failure happen: in the agent, in the tool call, or inside the remote server? Alex Volkov opens with this loss of visibility. Working on Weights & Biases’ Weave, he is accustomed to observing code he controls from end to end. Adding MCP capabilities pushes some of that execution beyond his instrumentation. Seeing the agent call a tool does not mean seeing what the tool does.
Benjamin Eckel, Dylibso’s co-founder and CTO, brings the perspective of mcp.run, which operates both MCP clients and servers. His team has assembled its own observability machinery, and he sees other teams independently solving the same problems. Each new integration can add another boundary that developers must instrument themselves.
That gap matters during an incident. Without knowing why, where, and how execution failed, a team has fewer ways to respond quickly. Eckel connects this to enterprise adoption: teams need to identify security and reliability problems before customers encounter them. MCP therefore needs to fit the observability platforms those teams already use.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A visible tool call can still hide its implementation
Volkov first demonstrates Weave’s MCP integration. The integration presented here is Python-based and assumes the developer controls both client and server. He highlights MCP_TRACE_LIST_OPERATIONS on both sides to include list operations, alongside visibility into MCP call durations. This flag is a list-tracing option within an instrumented application, not a complete setup recipe. In the displayed trace, arrows distinguish client traces from MCP tool calls, including a calculate-BMI tool.
The apparent success immediately exposes the remaining problem: the calculate-BMI call is visible, but its internal work is not. A duration tells the developer how long the call took without explaining which server operation consumed that time or failed. The integration is also specific to Weave’s Python SDK, leaving portability unresolved.
The Observable Tools Manifesto grows out of that limitation. It is a coordination effort among observability providers and tool builders to make MCP observability standardized and vendor-neutral. The next step is to connect MCP with an existing observability standard rather than make every provider invent its own tracing system.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Traces, spans, and a portable destination
MCP-powered agents are distributed systems. Their work crosses process and service boundaries, so the tracing techniques used for other distributed applications apply here. Eckel introduces OpenTelemetry, or OTel, through its basic building blocks. A trace groups the steps of an operation into a tree of spans. Each span records a step’s duration and attributes; the step can be as broad as an HTTP request or as small as a function call.
In the checkout API example, span size represents duration and placement represents the call structure. This lets a developer inspect both the sequence of work and the time spent in each part. The trace turns one overall checkout duration into an account of the operations that produced it.
Those spans need a destination. Eckel calls this a sink: a centralized place to receive telemetry, often packaged with a UI, dashboards, monitoring, and alerting. The useful separation is between producing telemetry and choosing the platform that stores and displays it.
| Component | Responsibility |
|---|---|
| Trace | Group related work into one operation |
| Span | Record a step’s duration and attributes |
| Sink | Receive telemetry for inspection and monitoring |
With standard instrumentation and export, changing the destination can largely become a configuration change. Application code need not be rewritten for each observability vendor. Volkov identifies Weave and Logfire among the LLM observability tools supporting OTel.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Following a fetch into the server
The checkout example becomes a distributed trace when the fraud service contributes its own spans. Exporting correlated spans to the same sink allows the platform to reconstruct the larger operation. The same principle applies when an agent calls an MCP server, but there is an additional question: who controls that server?
Eckel distinguishes two administrative domains, meaning ownership and operational control rather than DNS names or physical locations. A server hosted by GitHub or Stripe is outside the agent developer’s control. A server the developer owns in another data center can still be inside the same administrative domain.
| Server relationship | Visibility available to the client developer |
|---|---|
| Third-party server | Call boundary; internals depend on provider cooperation |
| Owned, instrumented server | Internal spans can join the client’s trace |
Consider the prompt, “Read and summarize the top article on Hacker News.” The agent reaches out to a remote fetch server, but when that server is outside the developer’s control, the displayed trace contains a single opaque service span.
If the developer owns the fetch server, its internal work can become visible. The server must produce spans, receive the caller’s trace context, and export its telemetry to the shared destination. A common sink alone is not enough: context propagation preserves the trace identifiers and parent relationships that let the sink connect the work. In Eckel’s illustrated fetch trace, HTTP requests take roughly 350 milliseconds, followed by processing to create Markdown. This is the timing of the example, not a benchmark for MCP.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Carrying context through MCP metadata
The TypeScript demonstration makes that connection through the MCP request’s _meta payload. When the client calls a tool, it captures the current span context and sends it with the request. The slide shows the context fields traceId, spanId, traceFlags, and isRemote: the identifying information needed to relate remote execution to the caller.
On the server, the receiving code extracts that context and uses it as the parent context for the server’s spans. For the Hacker News operation, the causal chain is:
- Capture the active context at the client’s fetch-tool call.
- Carry that context in the MCP request metadata.
- Extract it on the fetch server and establish the remote parent context.
- Create spans for server work under that context and export them to the sink.
The server’s HTTP and Markdown-processing work can then appear beneath the original call instead of as an unrelated trace.
This worked in Eckel’s demonstration, but it required a lower-level protocol interface rather than a convenient high-level SDK hook. His point is about the effort required at the time: developers should not have to work around SDK abstractions to preserve trace context. The working example he points to contains more setup than the slide, including the machinery needed to make the tracing flow run end to end.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Sending the TypeScript trace to Weave
The portability test is to take Eckel’s TypeScript agent and send its telemetry to Weave. The initial Weave integration was built into a Python SDK, but MCP does not require clients and servers to share a language, environment, or codebase. OTel extends that independence to the telemetry destination. Volkov reports adapting the TypeScript agent with little code change; the resulting Weave view shows green client traces alongside the work performed inside the server calls.
The export configuration has three responsibilities: choose Weave as the OTLP destination, authenticate the sender, and identify the destination project. Volkov gives wandb.ai/otel as the endpoint shorthand in the recording, then describes adding authorization headers and project routing. Treat that as the historical demonstration’s configuration, not a current executable endpoint; the maintained Weave OTLP guide is the setup reference. The architectural result is that the TypeScript agent can use Weave through a standard telemetry interface rather than the bespoke Python integration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A coding agent checks its own instrumentation
While adapting the agent, Volkov encounters a second use for MCP: giving the coding agent access to the observability system itself. He uses Claude Opus 4 to modify Eckel’s agent for Weave. The W&B MCP server, configured in his Windsurf environment, lets the coding agent inspect and summarize trace data. That creates a feedback loop between changing instrumentation and examining what it actually emits.
In Volkov’s account, the agent proceeds through a repair sequence:
- Modify the code and run it.
- Query Weave to check whether traces arrived.
- Discover that the traces exist but an input or output parameter is populated incorrectly.
- Find a support bot exposed through the MCP server and ask it for the missing information.
- Apply the guidance, then return to Weave to check the fix.
The documentation had not supplied the detail the agent needed. Instead of stopping at the malformed trace, it discovered another agent through a tool interface and used that interaction to continue the repair.
Volkov reports that he did not intervene at the keyboard during this sequence. The anecdote illustrates a useful role for observability in agent development: emitted telemetry can become feedback for the agent doing the implementation, not only a dashboard for the human reviewing it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Extending exports to profiles and tasks
Eckel then announces planned OTel exports from both sides of mcp.run’s offering:
- Profiles: Combine selected capabilities from multiple MCP servers into a single virtual server.
- Task: A single-prompt agent that can be triggered through a URL or a schedule and works with profiles.
Both are intended to export telemetry to OTel-compatible sinks, with Weave discussed as a destination. This is forthcoming support in the recording, distinct from the working context-propagation demonstration shown earlier.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Making traces understandable across platforms
The remaining adoption problem is not whether MCP observability can work. The demonstrations show that it can. It is whether teams can obtain that visibility without becoming tracing experts. OTel provides much of the infrastructure, but convenient SDK instrumentation and shared conventions are still needed. AI engineers can start by examining whether their traces cover the full execution chain; tool builders can make that coverage easier to obtain through higher-level SDKs. Volkov cites Arize’s OpenInference as a starting point for that work.
Transporting attributes does not establish what those attributes mean. A span allows user-defined metadata, but a sink needs a common interpretation to recognize an HTTP request with status 200 or an MCP tool call that ended in an error. Semantic conventions provide that agreement. Without them, telemetry can arrive successfully while each platform still needs its own interpretation of the same operation.
Eckel points builders toward the OTel GenAI semantic-conventions effort and asks platform providers to add OTel support, review RFCs, and contribute implementation experience. These are complementary tasks: instrumentation creates the spans, context propagation connects them, and conventions make their contents understandable across platforms. The ecosystem work is to make those pieces cooperate as a routine part of building an MCP application.
The closing invitation is to learn from people implementing these systems. Volkov emphasizes the practical knowledge gained in conversations with other builders. Eckel invites attendees to try mcp.run, while Volkov points to Weave’s MCP tracing material and the Observable Tools manifesto. Those resources and conversations are ways to turn a working demonstration into tooling that other teams can use without rebuilding the same integration themselves.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The initiative's principles and proposal for making MCP server execution visible through vendor-neutral telemetry.
Official MCP server for giving agents access to W&B experiments and Weave traces.
OpenTelemetry instrumentation and semantic conventions for AI applications, including MCP integrations.
Further reading
- How OpenTelemetry context propagation worksDocumentation
Explains how trace and parent-span identifiers connect execution across service boundaries.
Updates since the talk
Current Python setup for tracing MCP client and server operations, including optional list-operation tracing.
Endpoint, authentication, project routing, and exporter examples for sending OTLP traces to Weave.
- OpenTelemetry MCP semantic conventionsDocumentation
Developing conventions for MCP spans, metrics, transport attributes, and context propagation.
- MCP trace-context conventions: SEP-414Documentation
Documents standard trace-context keys carried in MCP request metadata.
Full transcript
Loading transcript…