AI Engineer Europe 2026
AI Didn’t Kill the Web, It Moved in! — Olivier Leplus (AWS) & Yohan Lasorsa (Microsoft)
Loading slides…
Read the talk
AI Didn’t Kill the Web, It Moved in!
Follow an e-commerce application from agent-assisted development and browser debugging through on-device summarization and proofreading to structured tools that let agents operate carts and forms.
From a talk by Olivier Leplus and Yohan Lasorsa
Before you start: Familiarity with browser DevTools, JavaScript application development, JSON schemas, and basic AI-agent tool calling will help readers follow the demonstrations.
An issue, a browser, a phone, and a human approval gate
Yohan Lasorsa gives his coding agent a deceptively simple assignment: inspect the open issues for Seine, an example e-commerce application, and implement the first one. Seine already has product descriptions and customer reviews; its outstanding issue requests a contact page. Without being told which tool to use, the agent reaches for the GitHub CLI, retrieves the issue, and starts implementing the feature. 3:11
The repository’s agent skills explain how it knows what to do. Each skill has a SKILL.md with a name, description, and task-specific instructions or command examples. The description tells the agent when that expertise is relevant, allowing specialized guidance to enter its context only when needed. Here, separate skills cover the GitHub CLI, frontend design, and Playwright-driven browser testing and video capture. 4:38
Lasorsa also builds two custom skills using a skill-creation skill: one establishes a public tunnel to his development machine, and another delivers the resulting URL to his phone through Telegram. Repository-level agent instructions turn those capabilities into an intended delivery procedure: 6:06
- Implement the requested feature.
- Record a short browser video with Playwright.
- Start the development server and expose a phone-accessible preview through the tunnel.
- Send the preview to Telegram.
- Leave the GitHub issue open until a human explicitly confirms the work.
The agent is still working, so Olivier Leplus steps in with Chrome DevTools MCP. This MCP server exposes Chrome’s browser controls and inspection capabilities as tools a coding agent can invoke. Configured through mcp.json, it provides actions including navigation, clicks, form entry, console and network inspection, Lighthouse audits, screenshots, and viewport resizing. 7:29
Asked to inspect the application’s main page, the agent checks package.json, starts the development server in a terminal, opens an automated Chrome session, navigates to the site, and captures screenshots. The browser is being controlled through exposed tools rather than through Leplus manually operating its interface.
Leplus next requests a comparison across fast, 3G, and 2G-style network conditions. The agent records an unthrottled baseline, runs throttled tests, and captures performance traces. Its analysis includes Largest Contentful Paint, cumulative layout shift, critical-path latency, and render-blocking opportunities. An oversized headphone image becomes particularly problematic on slower connections; proposed remedies include image prioritization, stylesheet and JavaScript improvements, and preloading relevant JSON. These are recommendations, not fixes already applied to the application. 10:45
Only after the DevTools interlude do the presenters return to the missing phone notification. The contact-page workflow stalled because the agent could not find the Telegram token and associated channel configuration. Lasorsa instructs it to use the existing .env file and retry. The notification finally arrives, carrying the recorded feature preview and a link to the newly created contact page, which he opens on his phone. The retry repairs delivery; it does not require rebuilding the feature. 13:48
The contact page is ready. Why hasn’t the phone buzzed?
In the Seine demo, delivery stalled because the agent could not find its Telegram token. Retry with the existing .env configuration.
Awaiting human confirmation
- Contact pageReady · kept
- Feature videoRecorded · kept
- Preview linkReady · kept
.envNot loadedNo notification
A delivered preview lets Lasorsa inspect the result from his phone, but delivery is not the human confirmation required to close the issue; no approval is shown. Telegram is simply one possible destination: the same workflow could send a preview to colleagues through Slack instead. 15:25
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Debug inside DevTools, then write the fix back to disk
Chrome DevTools MCP lets an external coding agent operate the browser. Lasorsa’s next demonstration reverses the vantage point: AI assistance embedded directly inside DevTools can inspect the current page without requiring the developer to copy browser errors into another chat window. These features must first be enabled under the DevTools settings’ AI-related options; they are not enabled by default in the demonstrated setup. 16:02
On Seine’s About page, the console contains a request blocked by cross-origin resource sharing, or CORS. Clicking the AI assistance icon produces an explanation of why the request failed and suggests possible fixes. In the Network panel, selecting a failed request adds that specific request to the assistant’s context; the resulting analysis identifies a bad request and suggests that the requested endpoint may no longer exist. 16:58
Lasorsa then switches to GitHub, records a trace in the Performance panel, and opens its Largest Contentful Paint breakdown. The assistant analyzes the recorded trace and points toward possible render-blocking work. Its response varies between runs: earlier attempts identified CSS more explicitly, while this run provides a less detailed explanation. The trace is an observed debugging input, not evidence that a particular optimization has already improved performance. 18:59
Back on Seine, Lasorsa selects the About-page h1 in the DOM inspector and asks the assistant to replace its plain heading with a gradient matching the site’s existing CSS variables. After he authorizes the modification, DevTools updates the rendered page. That initial edit is temporary: refreshing could discard it because the underlying source file has not changed. 20:33
The Unsaved Changes panel exposes the generated CSS, and Apply to Workspace can persist it. Adding the application’s source folder to the DevTools workspace allows the browser to write the CSS modification back to the project instead of leaving it as an ephemeral page tweak. Leplus notes the familiar failure mode this addresses: after experimenting extensively in DevTools, developers often cannot remember which exact edits must be copied into their stylesheet. 22:01
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run review summarization on a browser-managed local model
Leplus next moves AI from developer tooling into the application itself. Rather than sending every feature request to a remote provider, browser-native AI APIs can expose a model running locally on the user’s machine. The demonstrated family includes summarization, writing, rewriting, and general prompting, but support is experimental and depends on the browser and its configuration. 23:24
The first application feature condenses Seine’s customer reviews using the Chrome Summarizer API. Its setup has several independent choices:
- Availability: Check whether this browser exposes
Summarizerbefore creating a session. 24:19 - Shape and length: Choose key points, a short overview, a teaser, or a headline, and set the desired length.
- Language and context: Specify expected input and output languages, then explain that the supplied data contains product reviews.
A download monitor tracks whether the underlying model must first be installed. Leplus says the demonstrated model occupies approximately four gigabytes, is downloaded once and shared across websites, and can be removed by Chrome if device storage becomes constrained. Local inference therefore avoids a remote inference request in this example, but it still depends on browser support, an initial model download, adequate device storage, and local compute. 26:04
After setup, the application passes the review data to the summarizer and renders the returned summary. Because the model is already present on Leplus’s machine, the download indicator jumps immediately from zero to complete, and the interface produces a summary mentioning the headphones’ sound quality and battery life. 26:35
The demonstrated setup also requires enabling relevant Chrome flags for the Gemini-backed APIs. Chrome’s on-device internals provide a separate inspection surface for loading models, experimenting with image or audio inputs, adjusting generation settings such as top-k and temperature, reviewing event logs, and inspecting token usage associated with individual API calls. These diagnostics make the local model observable rather than treating browser-side inference as an invisible black box. 27:21
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Return corrected text and the exact edit locations
The Chrome Proofreader API follows the same basic lifecycle: first check whether the browser supports the feature, create a proofreader, attach a model-download monitor, and specify the expected language. Leplus configures English and connects proofreading to the product-review input. 28:15
When the review field loses focus, the application sends its contents for analysis and replaces spelling or grammatical mistakes with corrected text. A second test shows that the result includes more than the revised sentence: it also contains individual corrections with start and end positions identifying where the original input changed. An interface can therefore present corrections at the relevant character locations instead of silently replacing the entire review. 29:04
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate structured reviews from images—and inspect the result
Can Seine generate a product-review title and description directly from an uploaded photograph? Lasorsa turns to the general-purpose Chrome Prompt API, rather than the preceding task-specific summarizer and proofreader. He creates a browser language-model session configured for text and images, then prepares instructions asking the model to describe the product’s condition and imagine how its recipient felt. 30:17
The written instructions and uploaded image are combined in a single user message passed to session.prompt. To make the response usable by the application, Lasorsa constrains it with a plain JSON Schema describing an object with two string-valued properties: 31:30
json
{
"type": "object",
"properties": {
"title": { "type": "string" },
"description": { "type": "string" }
}
}
Those structured values can populate the review form directly.
Lasorsa uploads a headphone photograph and initially says the headphones appear to be in good shape. A warning then indicates that he has not specified the request language. Nevertheless, the model returns JSON describing a devastatingly damaged, broken headset and a disappointed recipient, and Seine fills its review fields with that generated assessment. The contradiction matters: the returned text is the model’s interpretation, not verified evidence that the headphones were actually damaged. 32:50
The demonstrated inference runs locally on the client inside the browser, without an external model request for each call. That does not eliminate the browser-managed model download, device-resource requirements, browser-specific availability, or configuration dependencies discussed earlier. Audio is mentioned as another possible input category, but this demonstration uses text and an image. The APIs were experimental, and the newly surfaced language requirement had not appeared when the presenters rehearsed roughly a week earlier. 34:07
Leplus also imagines a Pictionary-style application comparing a drawing with an image and producing a similarity percentage. That is a suggested application possibility, not a demonstrated Seine feature or a measured accuracy result. 35:30
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give agents a map before asking them to act
Agents can also become users of the finished website. Human usability and search-engine discoverability are no longer the only interface concerns: an application may need to make its content legible to software that is trying to locate relevant information before taking action. 36:02
Lasorsa introduces the llms.txt proposal by analogy with robots.txt and site maps. The proposed text file gives an agent a concise, structured map of useful material instead of requiring it to traverse every ordinary webpage. An Angular documentation example contains Markdown links directing an agent toward relevant documentation, such as pages covering animation. 36:41
A fuller variant, llms-full.txt, gathers the underlying documentation into a single text resource. In the Angular example, the file includes extensive current documentation and code examples. This addresses a practical mismatch: a coding model’s training data can lag behind framework releases, leaving it prone to retrieving outdated patterns or old AngularJS examples instead of current framework guidance. 38:22
Supplying the complete documentation gives the agent an up-to-date reference in its immediate context rather than relying entirely on whatever version happened to appear in its training data. The same pattern can apply beyond framework documentation to other information a website wants agents to discover and understand. 39:25
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Expose cart actions and existing forms as explicit agent tools
An agent trying to add an item to Seine’s cart should not have to guess which pixels represent the right button. WebMCP, highly experimental at the time of the demonstration, proposes exposing an application’s own actions as structured tools. That differs from Chrome DevTools MCP: the earlier server gave a development agent browser-inspection and testing tools, whereas WebMCP makes page-owned business operations available to software acting for a user. 40:14
Without those explicit actions, browser agents commonly inspect screenshots or the DOM, identify a visible control, and imitate human clicks or form entry. A named application tool instead supplies the operation’s purpose and expected arguments directly, avoiding the need to infer business intent from presentation.
The Add to Cart button already calls Seine’s existing addCart function. Leplus wraps that same behavior in a tool with a name, description, and structured arguments for an item name and quantity. The execute handler extracts those arguments and reuses addCart(item); because the existing function accepts only one item at a time, the handler loops once for each requested unit rather than inventing native quantity support. 42:05
Pseudocode
tool add_to_cart(item, quantity):
repeat quantity times:
addCart(item)
return confirmation
Leplus registers the tool on the page’s navigator object, where a browser extension discovers the available action.
The extension invokes the tool with water bottles and a quantity of five; five bottles are added through the existing cart behavior. Leplus then tries a separate demonstration from his development environment, asking an IDE-connected agent to add three laptops. It identifies the registered cart tool, calls it with quantity three, and updates the page. The extension and IDE are the clients actually demonstrated; an agentic browser invoking the same application actions on the user’s behalf remains the forward-looking goal. 44:23
Not every existing operation needs a separately handwritten JavaScript tool. Leplus annotates Seine’s existing product-review form with a tool name and description, allowing its inputs to become structured arguments automatically. Explicit parameter descriptions can clarify individual fields, but nearby HTML labels supply defaults when those annotations are absent. The generated schema exposes a rating from one to five, a review title, review text, and an optional photograph. 46:10
The first review invocation does not visibly work. Leplus checks whether his file was saved and retries with a rating of four, title “awesome product,” and review text “Love it.” This time the form fills successfully. 48:13
Filling a form is not submitting it. Ordinary execution leaves the populated review as a draft; adding the demonstrated autosubmit annotation changes the operation so it fills and submits the form without another human interaction. The adjacent controlled comparison holds the review values constant to isolate that consequential difference.
A named tool changes the application itself.
Use the demo’s five water bottles and review values in fresh local examples. WebMCP was highly experimental at the time of the talk.
Add five water bottles
Tool arguments · unchanged{
"item": "water bottle",
"quantity": 5
}addCart(item) × 5Empty
Fill only
Same blank starting formNot submitted
Fill + submit
Same blank starting formNot submitted
Leplus compares the coming adjustment to responsive design: websites once had to adapt to mobile visitors, and they may likewise need to accommodate agentic browsers. The forecast is not a claim that a stable, universally available browser interface already exists; WebMCP was changing rapidly and remained highly experimental. The practical direction is to preserve the human-facing website while making meaningful application actions explicit to software working on a user’s behalf. 50:10
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Run and inspect the shared e-commerce example used for reviews, browser-native AI and application tools.
Inspect the task-specific skills behind the issue-to-preview workflow instead of rebuilding the demonstration from a generic agent tutorial.
The official server exposes browser debugging and inspection tools to the coding agent demonstrated in the session.
Follow the browser-native summarization API used to condense product reviews, including availability checks and model setup.
Read the proposal for making application actions callable through structured tools instead of forcing agents to infer every UI operation.
Further reading
A related project co-maintained by Olivier Leplus demonstrates browser automation with vision, tools and human approval using Amazon Bedrock.
Related talks
- The Current State of Browser Agents
Compare agents that operate browser interfaces with applications that expose explicit tools.
Full transcript
Loading transcript…