AI Engineer Summit 2025
Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work
Loading slides…
Read the talk
Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work
A browser agent can combine page interaction, structured extraction and API calls, turning scattered tabs into spreadsheet data, research inputs and actions.
From a talk by Arjun Chintapalli and Bhavani Kalisetty
When the browser becomes the bottleneck
How much work remains after you find the information you need on a website? Someone still has to copy it into Google Sheets or a CRM, reconcile it with data from an API, and repeat the process on the next page. rtrvr.ai, pronounced Retriever, brings a natural-language agent into a Chrome side panel to combine browser actions, structured extraction and API calls. Arjun Chintapalli and Bhavani Kalisetty demonstrate how those capabilities fit together.
The bottleneck is more than manual entry. Outsourced scraping introduces cost and reliability concerns; traditional RPA bots can break when a website changes. Meanwhile, information available only through a website remains separate from information available through an API. An agent operating alongside normal browsing offers a way to connect those sources and move results into a working spreadsheet.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Check page state, then act
The first task is small but conditional: find the Latent Space podcast page on LinkedIn and follow it, unless the account already follows it. After the instruction is submitted, the agent fills the search field and inspects the page. It recognizes that the account is already following the page, so the requested outcome requires no new follow action. The instruction describes a desired state, including when to leave the page unchanged.
The next example moves from interaction to structured output. A request specifies data to extract for every article on a page, and the export operation writes the results to Google Sheets. These capabilities can be composed: first perform a task on the page, then extract the resulting data. Chintapalli estimates that this particular page extraction probably cost less than one cent; the demonstration does not establish a measured cost or a general per-page price.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Spread the work across tabs
A spreadsheet can also supply the input. Given a Google Sheet column of URLs, the agent can open those pages, interact with them and extract their contents. In the arXiv example, the request asks for specified fields from the first five PDFs in a search. The agent splits the work into subtasks, opens new tabs and processes them independently and simultaneously before writing results to Sheets. Applying to LinkedIn jobs is offered as another possible use of the same tab-based action pattern.
For product comparison, the user instead selects three Amazon product tabs. Leaving the prompt empty lets the agent infer likely comparison fields and collect them into a spreadsheet. Explicit instructions are still useful when the desired output goes beyond those inferred fields: the demonstration also extracts source image URLs so the products' images can be compared alongside their other data.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Change the review order before extracting
The Amazon review example makes the dependency between action and extraction concrete. The pages initially show top reviews, but the request asks for the most recent review. Extracting the current contents immediately would answer the wrong question. The workflow is:
- Select the product tabs to process.
- In each tab, change the review dropdown from top reviews to most recent.
- Extract the details of the most recent review from the updated page.
The agent performs the actions across the selected tabs in the background. Chintapalli points out the dropdown changing on the pages, then reports that the recent reviews have been extracted. The page state is part of the extraction task, not merely a source to read once.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use selected documents as research context
The next workflow starts with design documents already open in the browser. To prepare a quick summary for a colleague, the user selects the documents and requests their key points and summaries. Selection defines which sources the agent should read; the input can mix Google Docs, PDFs and Google Sheets. The resulting response distinguishes the individual documents, including one concerning LLMs with memory. This is a synthesis task over a chosen collection, rather than an instruction to search the web indiscriminately.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let missing fields guide navigation
Company research extends the task beyond the pages initially selected. Kalisetty asks for strategy, features and pricing across Retriever, Bardeen and Browse AI. The agent first generates an extraction schema from the request. Starting from the companies' homepages, it then navigates to pricing pages to locate information missing from the initial view. The requested fields therefore guide both the output structure and the browsing work needed to populate it.
Kalisetty positions this as an early deep-research capability relative to OpenAI's Operator. The practical feature in the demonstration is control over the research inputs: users can select documents they have access to and specify the URLs or websites the agent may visit. The collected company information is written to Google Sheets.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recover from an unsuitable starting URL
The stock-research example begins with rows in a Google Sheet. The request asks for a P/E ratio from Yahoo Finance, revenue for the last two years and a new field computing revenue growth from those values. This combines three operations: read the existing stock records, retrieve additional financial data and derive a value from the retrieved numbers.
One supplied URL points to a company's website rather than Yahoo Finance. The agent recognizes that the starting page does not match the requested source and navigates to Yahoo Finance to continue the extraction into Sheets. The recovery illustrates how the task description can override an unsuitable starting URL. It does not establish the accuracy of the financial values or demonstrate an investment decision.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn spreadsheet rows into WhatsApp messages
Browser interaction can feed an API operation rather than end with an export. Retriever's dynamic function calling lets users provide information about a tool instead of depending entirely on a fixed catalog of vendor-built connectors. The broad promise is integration with arbitrary third-party APIs; the concrete example here uses a configured WhatsApp messaging tool.
The selected spreadsheet contains two customers' phone numbers. Kalisetty asks the agent to send messages to those numbers. It identifies each number and invokes the WhatsApp tool, then reports that all messages were sent. Kalisetty opens WhatsApp to check the result and points to a received test message timestamped 2:32, matching the demonstration's local time. That visible receipt supports delivery to the displayed test number, separately from the agent's broader success report.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate a representation of the data
The messaging example suggests broader social-communication workflows across Instagram, Facebook and WhatsApp, though those additional deliveries are not demonstrated. The next demonstrated tool moves in a different direction: a graph bot, described as a mini agent inside Retriever Agent Studio, takes existing data and generates an analysis graph on demand. The same workflow that collects structured information can therefore also produce a representation intended for a person to inspect.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read webpage text inside the user's session
The architecture discussion separates how an agent observes a page from where its browser runs. Kalisetty contrasts Retriever's text-based approach with the screenshot-based interaction used by the launch versions of Operator and Claude computer use, and describes Google Mariner as hybrid. Screenshot-based observation captures the page visually as the agent acts or scrolls; Retriever instead supplies webpage text to the model. The speakers attribute lower hallucination and cost to that choice, but provide no controlled comparison establishing those advantages.
The second distinction is the browser session itself:
| Deployment | Relevant context | Infrastructure concern raised in the talk |
|---|---|---|
| Cloud browser | A separate session may show different content | Proxy routing and IP infrastructure |
| Local extension | The user's existing browser session | Uses the browser already on the user's machine |
Kalisetty names Browserbase while discussing cloud-hosted browsers and argues that maintaining the necessary network routing adds expense. The useful distinction is session context and deployment overhead; the talk does not measure their relative costs.
The extension can process active and background tabs while using content the user can already access. The speakers state that Retriever does not require users to share passwords and does not store them. Login-walled and subscribed content is available through the existing authenticated session; this is not a demonstrated universal bypass of paywalls or Cloudflare protections. Nor must every cloud agent expose passwords to its model: Operator documented manual takeover for login without collecting or screenshotting information entered during that takeover.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use text to work across background tabs
Chintapalli connects text-based observation to the earlier multi-tab demonstrations. With the page text directly in context, the model can use that content without requiring a screenshot of each background tab. In the described implementation, those tabs need not be visually rendered for the agent to act on them. This supports both independent work in parallel and actions informed by context from multiple tabs.
That is a specific advantage for background-tab interaction inside one local browser, not a universal prohibition on parallel vision-based agents. Operator's launch documentation also described simultaneous tasks in separate conversations. The architectural question is how the system obtains observations and isolates concurrent work, rather than whether vision alone makes concurrency impossible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Decompose repetitive work into smaller tasks
The product focus is repetitive knowledge work rather than consumer tasks such as booking flights or restaurants. The client-side Chrome extension is presented as a way to reduce hosting overhead and reach local or login-walled sites through the user's browser. This is the architecture shown in the recording; today's Retriever offering also includes cloud execution and other access methods.
Task decomposition is the corresponding reliability strategy. Instead of keeping a long sequence of actions on one tab, Retriever distributes subtasks to new tabs. Chintapalli argues that this reduces failures and avoids compounding errors, although no benchmark establishes a lower failure rate. For integrations, the parallel scaling strategy is to let users define, configure and share function calls, expanding the available tools without requiring the vendor to build every connector itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build datasets across volunteers' browsers
The closing proposal extends extraction beyond one person's workflow: an AI-powered exchange in which people collaborate through their local laptops to construct datasets. Consider a collection of local government events across the San Francisco Bay Area. Gathering it could require visiting hundreds or thousands of websites, making centralized collection costly. Chintapalli proposes that volunteers contribute extraction work through their extensions, sharing the effort of building otherwise difficult-to-assemble datasets. This exchange is a long-term goal, not a deployed capability demonstrated in the talk.
The recording closes with an invitation to download the extension from rtrvr.ai or use the on-screen QR code. The immediate entry point is an agent working beside the user in the browser; the proposed destination is collaborative collection across many such browsers.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Product homepage with access to the Chrome extension and current automation offerings.
Operator's launch explanation of screenshot-based browser interaction, simultaneous tasks and user takeover.
Anthropic's explanation of screenshot interpretation, cursor control and early computer-use limitations.
Web scraping and workflow automation for lead research, enrichment and data export.
No-code tools for extracting and monitoring website data.
Updates since the talk
Current guide to custom tools, authenticated webpage execution and spreadsheet-driven tool calls.
Full transcript
Loading transcript…