AI Engineer World's Fair 2025
MCP Agent Fine-Tuning Workshop - Ronan McGovern
Loading slides…
Read the talk
MCP Agent Fine-Tuning Workshop
A browser agent’s useful training data includes its tools, reasoning, actions and observations. Ronan McGovern follows that record from collection through Qwen3 adapter training and back to inference.
From a talk by Ronan McGovern
Before you start: Familiarity with Python, chat-model APIs and basic model fine-tuning will help you follow the agent and notebook examples.
What should a browser agent learn from a successful run?
How do you turn a successful browser-agent run into training data for a smaller model? The final answer alone leaves out much of the work: choosing a tool, supplying its arguments, interpreting the result and deciding what to do next. Ronan McGovern’s workshop follows that complete sequence—collect good MCP agent traces, save their tool definitions and conversations, fine-tune Qwen3, then run the trained model to inspect its behavior.
The workshop materials live in the mcp-agent-fine-tune directory of Trelis Research’s AI Worlds Fair 2025 repository. The supplied task environment is browser navigation. MCP integrations for Stripe, GitHub or Gmail could replace it, but starting with the browser keeps the collection and training pipeline consistent while you learn it.
MCP connects tool discovery to tool execution. A server supplies descriptions and input schemas so the model can decide how to use its tools. When the model requests an action, the agent routes that request to the server. The server executes it and returns a result, sometimes with guidance for the next action. That observation enters the conversation, allowing the model to make another call or answer the user.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Translate between MCP, the API and model text
The model sits behind an OpenAI-compatible endpoint. This gives the agent a familiar request and response interface, but MCP schemas and generated model text still need translation at three boundaries.
| Boundary | Required translation |
|---|---|
| Tool discovery | MCP metadata → API JSON tool definitions |
| Model output | Generated tool-call text → structured call |
| Tool execution result | MCP response → conversation tool message |
The workshop’s Qwen model emits calls in Hermes format. A parser detects those calls in the generated tokens and exposes structured arguments through the API. The parser name describes the output convention; it does not require the model itself to be named Hermes.
The prompt makes this convention explicit. Its system portion introduces the available tools and instructs the model to put JSON function calls inside XML-style tool_call tags. The user then asks it to navigate to trelis.com. The assistant can produce reasoning followed by a browser call; after receiving the page, it can reason again and return ordinary answer text. A generated call is a request for execution, not an executed browser action. The surrounding agent must recognize it and carry it out.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Serve a teacher that exposes reasoning
After cloning the repository into Windsurf, McGovern opens the README’s data-collection section; fine-tuning comes later in a notebook. For collection he chooses Qwen3-30B-A3B, a mixture-of-experts teacher described in rounded terms as 30 billion total parameters with 3 billion active. The companion command selects its FP8 variant. He recommends keeping the teacher and student in the same family, although a different stronger model could supply demonstrations. The practical requirement here is access to the reasoning being trained on: OpenAI models do not expose their private thinking traces.
The demonstrated host is RunPod, using a linked affiliate one-click template and a vLLM Docker image. An H100 is one suggested GPU. Other hosting services can work, and local hosting depends on whether the serving application supports the required tool calls; McGovern leaves LM Studio support unchecked. The deployment settings each serve a distinct purpose:
- Reasoning extraction: enable reasoning and its parser so thinking tokens become a separate response field.
- Context budget: configure a maximum model length of 32,000.
- Tool selection: enable automatic tool choice so the model decides whether and which tool to call.
- Call parsing: select the Hermes parser to turn generated calls into structured API output.
The historical parser option is documented in vLLM’s versioned tool-calling guide. These are the workshop’s serving choices, rather than a version-independent deployment command.
Expose port 8000, apply the template overrides and start the server. McGovern estimates roughly three minutes for startup. The agent receives the model name and an OpenAI-compatible base URL containing the RunPod pod ID, exposed port and /v1 suffix. That connects the local agent workspace to the remotely served teacher.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Approve a navigation and read the page
Browser results can dominate the context. A navigation returns an accessibility tree: a structured text description of the page that the model can read. The agent’s --truncate option keeps only the beginning of that response. In the companion instructions, this limit is measured in characters, separately from the model’s token budget. It is a deliberately simple compromise: content farther down the page can disappear from the model’s view. A more complete agent needs a way to retrieve relevant content beyond that prefix.
Install uv if needed, then synchronize the project dependencies:
bash
pip install uv
uv sync
The agent starts the MCP servers listed in its configuration and loads their tools. The supplied Playwright MCP server provides browser navigation, history, tabs and link interactions. McGovern initially reports 25 tools. He estimates that open models may become confused beyond roughly 25–50 tools, while Claude might handle around 200; these are practical estimates, not measured capacity limits.
The first request is to navigate to trelis.com and read out the top two lines. Thinking tokens are hidden from the terminal to reduce clutter, but retained for logging. The model proposes navigation and waits for approval. McGovern keeps the browser visible so the audience can watch the action; --headless would run it in the background. After arranging the windows and submitting the request again, he approves the call. The browser opens, its accessibility structure returns to the model, and the assistant identifies both lines.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Save the whole run, then try a harder task
The agent logs two essential fields by default: messages and tools. The former preserves the conversation; the latter preserves the tool definitions available during that conversation. Here McGovern describes the saved inventory as 26 tools, including browser close and resize, rather than the 25 reported at startup. The companion configuration launches @playwright/mcp@latest, so it does not establish a fixed historical count.
The saved record contains the user request, assistant reasoning, navigation call, truncated page response, further reasoning and final answer. That is the behavior to preserve. McGovern keeps this successful trace for training a smaller model: the teacher already solved this example, whereas a less capable student may benefit from learning its sequence of decisions. The next collection step is to repeat the process and retain more good runs for upload to Hugging Face Hub.
The harder task is to find the third product on trelis.com, Training and Fine-Tuning, navigate to its page and report the feature section. McGovern first follows the route manually and notices that the product opens in a new tab. The agent then navigates, successfully opens the dropdown, but stops early with an answer and asks whether to click Learn More.
McGovern tells it to continue and explicitly reminds it to switch tabs. The agent clicks Learn More, opens the new tab and requests tab selection. He is unsure whether selecting that tab automatically obtains a fresh snapshot, but the subsequent answer retrieves the correct features: GitHub repository access and the ability to post issues. This is a successful guided run, with the corrective user turn forming part of the recorded conversation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn guided runs into curated demonstrations
One way to collect cleaner behavior is to supply a task-specific system prompt. It can explain which tools to use, how to reach the product and when to switch tabs. McGovern’s proposal is to use that guidance during collection, then omit the task-specific prompt from the training record while retaining the resulting trace. The goal is for the student to learn the demonstrated sequence without requiring the same detailed instructions each time.
He then opens the more complicated trace, leaving its tool definitions unchanged, and considers two editing approaches:
- Combine the interrupted sequence: remove the corrective user turn and combine the assistant sections, manually or with ChatGPT. Remove the intervening assistant answer content while retaining the reasoning needed for the continued action.
- Reduce the correction: retain a short instruction to proceed, but move the possible need to switch tabs into the assistant’s reasoning. This is the edit he demonstrates.
The resulting examples are curated demonstrations, including retrospective edits, rather than a collection consisting only of untouched autonomous successes. That distinction matters when interpreting what the student is being taught.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Unroll conversations into training prefixes
Uploading a trace means preserving both its conversation and its tool list. Unrolling also saves shorter conversation prefixes so intermediate responses can become training endpoints. McGovern illustrates a conversation with three back-and-forths:
| Training row | Conversation retained | Endpoint |
|---|---|---|
| Short prefix | First exchange | Earlier assistant response |
| Longer prefix | First two exchanges | Later assistant response |
| Full record | All three exchanges | Final assistant response |
His motivation includes reasoning retention: he describes the workshop template as including reasoning only from the most recent turn. Making an earlier response the endpoint lets that reasoning appear in its own training row.
This behavior depends on the uploader and template revisions. The current companion uploader adds prefixes at assistant messages immediately following user messages, plus the full conversation—not at every assistant tool step. The current Qwen3-4B chat template retains reasoning after the last real user query, which can span several assistant/tool steps. For a reproduction, inspect those boundaries together rather than assuming every tool call produces a row or that only one assistant message can retain reasoning.
The upload helper takes a repository ID and an optional unroll argument. Hugging Face authentication needs a token with write permissions. In the demonstrated upload, four collected examples become nine dataset rows: one contributes three rows and the other three contribute two each. The displayed dataset includes an ID, timestamp, model, messages, tools and a flag described as truncated or unrolled. Here truncation means shortening the conversation to a prefix; it is separate from shortening a browser’s accessibility response.
Inspecting the rows makes the difference concrete. A full record reaches the answer about the product page. A shorter record ends with reasoning and a tool call to select the tab. The reasoning remains in a separate field because the endpoint’s reasoning parser extracted it during collection. Training will later combine each row’s messages and tools through the student’s chat template.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the agent’s translation responsibilities explicit
The implementation walkthrough maps the visible browser behavior to a small set of runtime responsibilities. The agent configures its OpenAI-compatible client, boots the MCP services, discovers their tools and translates their schemas. An MCP tool’s description and input schema become an API function definition; the Python mapping has this shape:
python
def to_openai_tool(tool):
return {
"type": "function",
"function": {
"name": tool.name,
"description": tool.description or "",
"parameters": tool.inputSchema,
},
}
The reverse path takes a structured model call back to the appropriate MCP tool. Once execution finishes, the agent wraps the result as a conversation message the API can accept. These conversions are the code-level counterparts of the earlier architecture diagram.
The chat loop ties discovery, model requests, tool execution and subsequent model responses together. It keeps parsed reasoning separate from assistant answer content, while a trace-logging helper saves the details needed for training. Server configuration and command-line arguments control the environment without changing those core responsibilities.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Template the records and test the untuned student
Fine-tuning moves to a modified Unsloth notebook. After the Colab-specific installation, McGovern loads Qwen3-4B in 16-bit precision, without 4-bit weight quantization. Larger models may exceed the free GPU’s capacity; his alternative is a RunPod fine-tuning template with Jupyter, where the notebook can be uploaded. He postpones attaching LoRA adapters so he can first inspect the untouched student’s behavior.
The notebook loads the dataset and inspects messages, tools and the tokenizer’s chat template. That template is the serialization contract: it turns structured records into the single sequence the model consumes. It is also available in the model’s tokenizer configuration on Hugging Face.
After the runtime restart and weight download, the records are formatted into a text field. The central operation is:
python
def format_record(row, tokenizer):
return {
"text": tokenizer.apply_chat_template(
row["messages"],
tools=row["tools"],
tokenize=False,
add_generation_prompt=False,
)
}
The output starts with system-level tool instructions and definitions, then includes the user messages, assistant messages, calls and tool responses. Those tool definitions and page observations make the sequence long. Before training, the notebook checks the longest formatted row against its configured maximum length of 32,000.
The baseline probe uses the first dataset request: navigate to trelis.com. An output already visible in the notebook came from an earlier fine-tuned run, so McGovern reruns the missing dataset cells before starting the fresh baseline. He reports nine rows and a longest formatted-row length of approximately 18,000. The raw 4B student receives the navigation request with 26 tools available. Its output is streamed to inspect whether it chooses a tool; a 2,000-token generation limit could be consumed by thinking before a call appears.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Fit adapters within a small memory and data budget
With baseline generation underway, McGovern applies rank-32 LoRA adapters to attention modules and MLP layers. The base matrices stay frozen while the smaller adapter matrices learn the update. He also enables rank-stabilized LoRA. Although he describes it in terms of adapting the learning rate to adapter size, the precise change is adapter scaling: PEFT’s LoRA configuration uses α/√r for rank-stabilized LoRA instead of ordinary LoRA’s α/r. This does not automatically change the optimizer’s learning-rate setting.
The trainer consumes the previously prepared text field. Its intentionally small configuration is:
| Setting | Workshop choice |
|---|---|
| Device batch size | 1 |
| Gradient accumulation | 1 |
| Epochs | 1 |
| Warm-up | Remove for this short run |
| Learning-rate schedule | Constant |
| Optimizer | AdamW 8-bit |
With nine rows, one epoch, device batch size one and gradient accumulation one, the run has nine update steps. McGovern describes the learning rate as relatively high for the small model, without giving its numeric value aloud. The 8-bit optimizer saves memory; a larger GPU such as an A40 or H100 would allow a larger batch. With this tiny batch, the training loss will be noisy.
Before training, McGovern reports approximately 15 GB of device memory, with 10 GB already reserved. Returning to the baseline output, he finds that the untuned student has more or less chosen the correct action. Reading both requested lines is his prediction, rather than a completed browser rollout in this notebook. He expects harder multi-step tasks to remain difficult, pointing back to the assistance needed by the larger teacher.
The training display reports that 1.62% of the model’s parameters are trainable; the main weights remain frozen. The experiment is therefore updating a small adapter set, not retraining the entire student.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Export the student and design a meaningful evaluation
The notebook provides a route back to the agent environment:
- Save the trained model and tokenizer.
- Authenticate to Hugging Face.
- Merge the adapters into 16-bit weights for the recommended export, then upload the merged model and tokenizer to Hub.
- Replace the model repository name in the existing RunPod serving configuration with that uploaded repository.
This would expose the student through the same style of inference endpoint used during collection. McGovern describes these export and deployment options while training proceeds.
A falling training loss is only a first diagnostic. McGovern recommends collecting a few hundred examples, reserving an evaluation split and logging with TensorBoard. The displayed loss is bumpy, consistent with the batch size of one. He suggests a batch around 32 for a larger experiment, with gradient accumulation as another way to increase the effective batch. But using the entire tiny dataset in one update would leave almost no sequence of learning steps; more data is part of the solution, not just a larger batch setting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give reward training a useful starting policy
Could trace generation be automated, or replaced with reward-based training such as GRPO? McGovern recommends beginning with manually curated, high-quality domain traces and supervised fine-tuning. His rationale is about exploration: if the initial model rarely completes the task, it generates many unsuccessful trajectories and receives few positive rewards. Teaching useful behavior first can make subsequent reward training more productive.
For browser tasks, he proposes a simple source of verifiable rewards: choose obscure websites, record the correct text from their top lines, and ask the model to navigate to those pages and retrieve it. The dataset then contains a ground-truth answer against which the result can be checked. The essential requirement is a systematic way to produce correct targets and reward successful retrieval; the workshop itself remains a supervised fine-tuning exercise.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A correct tool call closes the loop, not the evaluation
Training finishes with a generally lower loss. McGovern reports peak memory use of 75% and doubts that a device batch size of two would have fit. The notebook starts post-training generation with about 2,000 tokens available. He explicitly allows for the possibility that this small, noisy training run could disturb an answer the baseline already got right.
The fine-tuned model emits the expected tool call. That is a useful integration check, but the untuned student also selected the basic navigation action, and the workshop presents no held-out score or measured improvement in complete agent success. The next evaluation is to serve the student and run actual browser rollouts in the earlier workspace. The displayed inference configuration uses temperature 0.6, consistent with Qwen’s thinking-mode guidance rather than a universal setting for reasoning models.
The practical extension is to add the MCP servers needed for a real task and curate more examples of its recurring action sequences. McGovern suggests that roughly 50–100 manually generated examples could improve common, narrow task paths even without reinforcement learning; that is his expectation, not a measured result from this run. The workshop repository provides the starting materials, and its two companion MCP videos cover server setup and building a custom server. The useful specialization target is a repeatable task whose successful decisions, tool calls and observations you can capture well enough to teach.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Agent scripts, trace collection instructions and links to the fine-tuning notebook, slides and introductory MCP videos.
Tokenizer configuration showing how tool definitions, reasoning, calls and responses become model input.
Browser automation server exposing navigation and page inspection through MCP and accessibility snapshots.
Further reading
Versioned documentation for tool-call parsers, including the Hermes parser option used in the workshop.
Original explanation of square-root rank scaling for LoRA adapters.
Full transcript
Loading transcript…