AI Engineer Summit 2025

Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work

Loading slides…

Read the talk

Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work

A browser agent can combine page interaction, structured extraction and API calls, turning scattered tabs into spreadsheet data, research inputs and actions.

From a talk by Arjun Chintapalli and Bhavani Kalisetty

When the browser becomes the bottleneck

How much work remains after you find the information you need on a website? Someone still has to copy it into Google Sheets or a CRM, reconcile it with data from an API, and repeat the process on the next page. rtrvr.ai, pronounced Retriever, brings a natural-language agent into a Chrome side panel to combine browser actions, structured extraction and API calls. Arjun Chintapalli and Bhavani Kalisetty demonstrate how those capabilities fit together.

The bottleneck is more than manual entry. Outsourced scraping introduces cost and reliability concerns; traditional RPA bots can break when a website changes. Meanwhile, information available only through a website remains separate from information available through an API. An agent operating alongside normal browsing offers a way to connect those sources and move results into a working spreadsheet.

Orange slide titled “Your Browser is the Bottleneck.” Four text blocks describe workflow problems, with the presenter visible at right.Open full source frame
Four browser workflow bottlenecks: manual data entry, offshore scraping, brittle RPA bots, and data silos.
0:010:36
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:01 · section reference included

Check page state, then act

The first task is small but conditional: find the Latent Space podcast page on LinkedIn and follow it, unless the account already follows it. After the instruction is submitted, the agent fills the search field and inspects the page. It recognizes that the account is already following the page, so the requested outcome requires no new follow action. The instruction describes a desired state, including when to leave the page unchanged.

The next example moves from interaction to structured output. A request specifies data to extract for every article on a page, and the export operation writes the results to Google Sheets. These capabilities can be composed: first perform a task on the page, then extract the resulting data. Chintapalli estimates that this particular page extraction probably cost less than one cent; the demonstration does not establish a measured cost or a general per-page price.

1:381:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:38 · section reference included

Spread the work across tabs

A spreadsheet can also supply the input. Given a Google Sheet column of URLs, the agent can open those pages, interact with them and extract their contents. In the arXiv example, the request asks for specified fields from the first five PDFs in a search. The agent splits the work into subtasks, opens new tabs and processes them independently and simultaneously before writing results to Sheets. Applying to LinkedIn jobs is offered as another possible use of the same tab-based action pattern.

For product comparison, the user instead selects three Amazon product tabs. Leaving the prompt empty lets the agent infer likely comparison fields and collect them into a spreadsheet. Explicit instructions are still useful when the desired output goes beyond those inferred fields: the demonstration also extracts source image URLs so the products' images can be compared alongside their other data.

3:093:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:09 · section reference included

Change the review order before extracting

The Amazon review example makes the dependency between action and extraction concrete. The pages initially show top reviews, but the request asks for the most recent review. Extracting the current contents immediately would answer the wrong question. The workflow is:

  1. Select the product tabs to process.
  2. In each tab, change the review dropdown from top reviews to most recent.
  3. Extract the details of the most recent review from the updated page.

The agent performs the actions across the selected tabs in the background. Chintapalli points out the dropdown changing on the pages, then reports that the recent reviews have been extracted. The page state is part of the extraction task, not merely a source to read once.

4:404:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:40 · section reference included

Use selected documents as research context

The next workflow starts with design documents already open in the browser. To prepare a quick summary for a colleague, the user selects the documents and requests their key points and summaries. Selection defines which sources the agent should read; the input can mix Google Docs, PDFs and Google Sheets. The resulting response distinguishes the individual documents, including one concerning LLMs with memory. This is a synthesis task over a chosen collection, rather than an instruction to search the web indiscriminately.

Google Docs displays the end of an AI web agent design document, while the sidebar shows an extraction request and a long response headed “Task Response.”Open full source frame
A document summary appears in the rtrvr.ai sidebar beside a Google Doc.
6:106:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:10 · section reference included

Let missing fields guide navigation

Company research extends the task beyond the pages initially selected. Kalisetty asks for strategy, features and pricing across Retriever, Bardeen and Browse AI. The agent first generates an extraction schema from the request. Starting from the companies' homepages, it then navigates to pricing pages to locate information missing from the initial view. The requested fields therefore guide both the output structure and the browsing work needed to populate it.

Kalisetty positions this as an early deep-research capability relative to OpenAI's Operator. The practical feature in the demonstration is control over the research inputs: users can select documents they have access to and specify the URLs or websites the agent may visit. The collected company information is written to Google Sheets.

7:087:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:08 · section reference included

Recover from an unsuitable starting URL

The stock-research example begins with rows in a Google Sheet. The request asks for a P/E ratio from Yahoo Finance, revenue for the last two years and a new field computing revenue growth from those values. This combines three operations: read the existing stock records, retrieve additional financial data and derive a value from the retrieved numbers.

One supplied URL points to a company's website rather than Yahoo Finance. The agent recognizes that the starting page does not match the requested source and navigates to Yahoo Finance to continue the extraction into Sheets. The recovery illustrates how the task description can override an unsuitable starting URL. It does not establish the accuracy of the financial values or demonstrate an investment decision.

8:278:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:27 · section reference included

Turn spreadsheet rows into WhatsApp messages

Browser interaction can feed an API operation rather than end with an export. Retriever's dynamic function calling lets users provide information about a tool instead of depending entirely on a fixed catalog of vendor-built connectors. The broad promise is integration with arbitrary third-party APIs; the concrete example here uses a configured WhatsApp messaging tool.

The selected spreadsheet contains two customers' phone numbers. Kalisetty asks the agent to send messages to those numbers. It identifies each number and invokes the WhatsApp tool, then reports that all messages were sent. Kalisetty opens WhatsApp to check the result and points to a received test message timestamped 2:32, matching the demonstration's local time. That visible receipt supports delivery to the displayed test number, separately from the agent's broader success report.

WhatsApp Web shows the Test Number conversation with repeated “Hello World” messages; the rtrvr.ai sidebar displays “All messages sent.”Open full source frame
WhatsApp displays test messages alongside the agent’s message-sending report.
9:429:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:42 · section reference included

Generate a representation of the data

The messaging example suggests broader social-communication workflows across Instagram, Facebook and WhatsApp, though those additional deliveries are not demonstrated. The next demonstrated tool moves in a different direction: a graph bot, described as a mini agent inside Retriever Agent Studio, takes existing data and generates an analysis graph on demand. The same workflow that collects structured information can therefore also produce a representation intended for a person to inspect.

11:0411:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:04 · section reference included

Read webpage text inside the user's session

The architecture discussion separates how an agent observes a page from where its browser runs. Kalisetty contrasts Retriever's text-based approach with the screenshot-based interaction used by the launch versions of Operator and Claude computer use, and describes Google Mariner as hybrid. Screenshot-based observation captures the page visually as the agent acts or scrolls; Retriever instead supplies webpage text to the model. The speakers attribute lower hallucination and cost to that choice, but provide no controlled comparison establishing those advantages.

The second distinction is the browser session itself:

DeploymentRelevant contextInfrastructure concern raised in the talk
Cloud browserA separate session may show different contentProxy routing and IP infrastructure
Local extensionThe user's existing browser sessionUses the browser already on the user's machine

Kalisetty names Browserbase while discussing cloud-hosted browsers and argues that maintaining the necessary network routing adds expense. The useful distinction is session context and deployment overhead; the talk does not measure their relative costs.

The extension can process active and background tabs while using content the user can already access. The speakers state that Retriever does not require users to share passwords and does not store them. Login-walled and subscribed content is available through the existing authenticated session; this is not a demonstrated universal bypass of paywalls or Cloudflare protections. Nor must every cloud agent expose passwords to its model: Operator documented manual takeover for login without collecting or screenshotting information entered during that takeover.

12:0512:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:05 · section reference included

Use text to work across background tabs

Chintapalli connects text-based observation to the earlier multi-tab demonstrations. With the page text directly in context, the model can use that content without requiring a screenshot of each background tab. In the described implementation, those tabs need not be visually rendered for the agent to act on them. This supports both independent work in parallel and actions informed by context from multiple tabs.

That is a specific advantage for background-tab interaction inside one local browser, not a universal prohibition on parallel vision-based agents. Operator's launch documentation also described simultaneous tasks in separate conversations. The architectural question is how the system obtains observations and isolates concurrent work, rather than whether vision alone makes concurrency impossible.

14:2614:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:26 · section reference included

Decompose repetitive work into smaller tasks

The product focus is repetitive knowledge work rather than consumer tasks such as booking flights or restaurants. The client-side Chrome extension is presented as a way to reduce hosting overhead and reach local or login-walled sites through the user's browser. This is the architecture shown in the recording; today's Retriever offering also includes cloud execution and other access methods.

Task decomposition is the corresponding reliability strategy. Instead of keeping a long sequence of actions on one tab, Retriever distributes subtasks to new tabs. Chintapalli argues that this reduces failures and avoids compounding errors, although no benchmark establishes a lower failure rate. For integrations, the parallel scaling strategy is to let users define, configure and share function calls, expanding the available tools without requiring the vendor to build every connector itself.

15:1215:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:12 · section reference included

Build datasets across volunteers' browsers

The closing proposal extends extraction beyond one person's workflow: an AI-powered exchange in which people collaborate through their local laptops to construct datasets. Consider a collection of local government events across the San Francisco Bay Area. Gathering it could require visiting hundreds or thousands of websites, making centralized collection costly. Chintapalli proposes that volunteers contribute extraction work through their extensions, sharing the effort of building otherwise difficult-to-assemble datasets. This exchange is a long-term goal, not a deployed capability demonstrated in the talk.

The recording closes with an invitation to download the extension from rtrvr.ai or use the on-screen QR code. The immediate entry point is an agent working beside the user in the browser; the proposed destination is collaborative collection across many such browsers.

Orange slide reading “rtrvr.ai: Unlock the Full Potential of Your Browser!” above a large QR code, with contact and support addresses at lower right.Open full source frame
The closing rtrvr.ai slide presents a QR code and contact information.
16:3116:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:31 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…