How to Build Financial Agents That Work Across the Web - Jay Sahnan & Arik Bird, Browserbase

Loading slides…

Read the talk

How to Build Financial Agents That Work Across the Web

Jay Sahnan and Arik Bird use Ramp, BILL and Stripe to explain three separate requirements for browser agents: reaching protected websites, adapting to changing interfaces, and turning web pages into structured data.

From a talk by Jay Sahnan and Arik Bird

At a glance

Ideas worth remembering

  • Ramp’s receipt workflow connects a card transaction to a receipt behind a vendor login: launch a browser agent, log in, capture the receipt and match it to the purchase.

  • Reliable hosted browsing involves browser identity, session continuity and process behavior as well as network access. Browserbase maintains a Chromium fork to change several of those signals.

  • Stagehand combines deterministic browser operations with natural-language control. In the demonstrated comparison, its reduced page representation uses roughly 4,000 tokens instead of 122,000 tokens of raw HTML.

  • Schema-based extraction turns page content into data another program or agent can consume. The closing demo expands this research from Chipotle’s website to multiple sources and then twenty companies, while leaving financial interpretation as a separate task.

Ramp: a transaction needs a receipt behind a login

A card transaction can arrive before the paperwork needed to reconcile it. Ramp’s receipt-fetching problem begins on the other side of that gap: someone has to visit a vendor’s portal, log in, find the receipt and connect it to the transaction. As Ramp grew, adding people to do those visits meant adding manual work alongside each increase in volume. Jay Sahnan and Arik Bird of Browserbase use this problem to begin their explanation of financial browser agents.

The browser supplies an agent with the same broad interface a person uses: it can read a page, interact with controls and take actions on websites that offer no suitable API or MCP connection. That matters for financial workflows because important information sits behind authenticated portals and anti-bot protections. Ramp, BILL and Stripe provide three examples of work that depends on reaching those parts of the web. 1:00

The flight booked for this conference makes the receipt workflow concrete. A purchase on a Ramp card triggers a browser agent. The agent visits Delta, logs in, captures the receipt and matches it to the card transaction. The useful result is the completed connection between two records: the purchase Ramp already knows about and the receipt held by the airline. Browser access lets the automation cross that gap.

Three steps describe signing into a merchant account, finding an order and downloading its receipt, and matching the receipt to a transaction. Beside them, a mock transaction marked Receipt missing appears above an order-history page with a Download receipt button.Open full source frame
The same $428 flight appears on the card statement and in the merchant’s order history. Ramp’s agent signs in, downloads the receipt and matches it to the transaction.

Running that sequence once is only the beginning. Running it across many transactions brings anti-bot systems into the critical path: a blocked browser cannot fetch the receipt, however capable the agent directing it may be. The next engineering problem is therefore how to make browsers run reliably away from a person’s laptop.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

Why moving Chromium into a data center changes the visit

A familiar laptop browser carries more context than the page currently open in it. Websites can encounter a browser fingerprint comprising its version, operating system, fonts, screen size and time zone. Cookies and previous logins supply continuity, while network reputation and navigation patterns provide further signals. In the presenters’ explanation, these signals together help a website recognize a familiar visitor. 3:40

Starting fresh headless browsers in a data center changes that picture. The presenters describe launches with changing fingerprints, fresh cookies and renewed login requirements, arriving from an address recognizable as data-center infrastructure. A proxy can change the apparent location or reduce some CAPTCHA challenges, but it addresses the network part of the visit. It does not by itself align the browser’s identity or change how Chromium runs.

Chromium also contains several interacting systems: a page engine handling HTML and JavaScript, networking that sends and receives requests, and device and graphics behavior. Browserbase’s difficulty was maintaining automation across that whole browser as upstream code changed. The presenters describe changes arriving every two weeks. Their response was to fork Chromium and maintain a version for their hosted environment—a continuing maintenance commitment rather than a one-time launch configuration.

Three changes illustrate what Browserbase calls its “verified browser”:

  • Automation indicators: The fork changes the reported WebDriver automation flag from true to false throughout the relevant code paths.
  • Browser identity: It presents itself as Chrome rather than headless Chrome. The work involves finding identity signals spread through the implementation.
  • Process lifetime: At the end of a task, Browserbase closes the windows while leaving Chromium running, instead of immediately killing the browser process.

These changes address different parts of the hosted browser’s behavior. They do not establish that every protected site will accept every session; later in the demonstration, the presenters still describe encountering and solving CAPTCHAs.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:40 · section reference included

From one live browser to a hundred concurrent sessions

The first demonstration launches a hosted browser and displays its live view in an iframe. It visits Ramp, then stops. The next launch repeats the operation with 100 concurrent browsers visiting different websites. At this stage, the demonstration exercises browser infrastructure: the browsers are visiting pages, without yet automating the business workflows. 7:49

How can an audience tell that these are separate, usable sessions? A QR code assigns attendees a random browser from the hundred. Several confirm that a browser has loaded, and the iframe allows them to click and interact with it. This makes the distinction between a remote browser and its streamed view tangible: the browser runs in Browserbase’s cloud while the attendee controls it through the embedded interface. The presentation’s Wi-Fi and an ngrok tunnel through the presenter’s laptop complicate delivery of that view.

The Ramp case closes with the business result reported by the presenters: thousands of deployable browser agents, more than 30,000 manual data-entry hours saved, and a fivefold increase in scale. The recording does not define the period or workload behind the fivefold figure, so it is best read as a reported customer outcome rather than a throughput benchmark. The practical benefit is that receipt collection no longer requires a person to perform every portal visit.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:49 · section reference included

BILL: one agent across thousands of different portals

BILL introduces a different failure mode. Its payments cross thousands of vendor portals, each with its own checkout flow, payment form and changing interface. Supplying enough browsers does not tell an agent which field to fill or which button completes the next step. The proposed goal is “one agent, thousands of different portals,” avoiding a separate hand-maintained automation for every vendor. 10:18

Stagehand is Browserbase’s open source SDK for controlling those browsers, built on top of Playwright. The brittle-script example is a form whose label changes from “first name” to “name.” A script tied to the old label, XPath or selector can lose its target even though a person still understands what the field means. The failure comes from the particular assumption encoded in the script; the interface can remain usable while that assumption stops holding.

Stagehand supports two ways to express control:

  • Deterministic operations: Use explicit browser operations such as page.click where the target and interaction are already known.
  • Natural-language actions: Describe the intent—fill out a form, enter a name or password, or find a button—and let the model identify the relevant control from the current page.

These approaches can coexist. The natural-language path gives the automation a way to interpret an interface when a fixed instruction would require more site-specific knowledge.

For that second path, Stagehand takes the page’s DOM and accessibility information, passes a representation to an LLM, and uses the model’s interpretation to determine where to click. The instruction supplies the goal; the current page supplies the available controls. That shifts some work from preserving a prewritten selector to resolving intent against the interface encountered during execution. It also introduces a cost: the model needs enough page context to make that decision.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:18 · section reference included

Send a smaller page representation, then move control closer

What does the model need to see to choose a control? The comparison shows raw HTML at 122,000 tokens beside Stagehand’s reduced accessibility representation at roughly 4,000 tokens for the same example. The reduced representation still provides direct selectors for acting on the page. This is a comparison of the inputs chosen for this demonstration, rather than a requirement that every Playwright-based agent send raw HTML. 13:23

The data flow explains why that reduction matters: page structure becomes model input, the model identifies a target, and Stagehand uses a selector to act on the browser. Sending fewer tokens lowers the amount of input being paid for and reduces the context the model has to process. The presenters also report quicker results, without providing a measured latency comparison. The demonstrated token counts apply to this page, rather than establishing a fixed compression ratio for all websites.

A second optimization changes where browser control runs. Repeatedly sending page content between a data-center browser and an external controller creates network round trips. Browserbase places Stagehand beside the hosted browser, describing this implementation as a Chrome extension. Localizing that interaction removes the controller-to-browser round trips the presenters identify. It does not imply that page loads or model processing take no time.

The two layers now have distinct jobs. Browserbase supplies hosted browsers at scale and works on access to protected sites. Stagehand decides how to interact with each page, using explicit operations or model-assisted interpretation. BILL’s heterogeneous portals need both: many available sessions and a way to control interfaces that do not share one fixed layout.

How page information becomes a browser action

Current web pagehosted-browser — DOM and accessibility information describe the interface.
Stagehand representationhosted-browser — Roughly 4,000 tokens in the demonstrated example, compared with 122,000 tokens of raw HTML.
LLM interprets the goalUses the page representation to determine the relevant control.
Stagehand executeshosted-browser — Uses a direct selector to click the target.
The model receives a reduced representation of the page. A selected target then becomes an action in the hosted browser; the model’s location is unspecified.
Read the diagram as text
  • Current web page. hosted-browser — DOM and accessibility information describe the interface.
  • Stagehand representation. hosted-browser — Roughly 4,000 tokens in the demonstrated example, compared with 122,000 tokens of raw HTML.
  • LLM interprets the goal. Uses the page representation to determine the relevant control.
  • Stagehand executes. hosted-browser — Uses a direct selector to click the target.
  • Current web page → Stagehand representation: Reduce page structure.
  • Stagehand representation → LLM interprets the goal: Supply page context.
  • LLM interprets the goal → Stagehand executes: Identify where to click.
  • Stagehand executes → Current web page: Click the selected control.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:23 · section reference included

Stripe: turn browser research into structured information

Stripe’s example moves from payments to merchant verification. The presenters describe people visiting Secretary of State portals to collect information needed to check whether businesses were real and suitable to onboard. Browser agents automate those visits for know-your-customer, or KYC, verification. This case establishes the collection workflow; it does not demonstrate that extracting a record alone completes every judgment involved in onboarding. 14:52

Developers can build such workflows from three Stagehand primitives, rather than handing the entire task to a general browser agent:

  • observe: Find a relevant control, illustrated by locating the “Learn More” button.
  • act: Perform an interaction, illustrated by clicking “Login.”
  • extract: Convert information on the page into structured output described by a Zod schema.

The examples separate finding a target, interacting with the interface and reading useful data from it.

Extraction is the bridge from a website to a downstream program or agent. Web pages contain messy, unstructured information; a schema describes the shape of the data the application wants back. Instead of asking the next component to interpret the whole page again, the workflow can pass it a structured result. The schema organizes the output, while the task still determines which information matters.

What does that look like on a small example? The demo launches a Browserbase session, visits Chipotle’s website and uses Stagehand’s extract operation to find out what the company sells. The returned information identifies Chipotle Mexican Grill and includes lime chips. This is a modest but useful observable change: content on a company website becomes a result the application can read. It is a research demonstration, separate from the Stripe portal workflow.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:52 · section reference included

Expand the same research from one page to twenty companies

The next run keeps Chipotle as the subject and expands the research to four sources: its company website, investor-relations page, SEC filing and New York Stock Exchange listing. The point of the expansion is that each source presents a different interface while contributing information about the same company. Separate browser views show the sources loading as the research proceeds. 17:06

The live views lag behind the work happening in the browsers. The presenters explicitly distinguish execution from its streamed display: some operations have already happened before they appear on screen. At scale, the live view is primarily a development aid. Watching every session would reintroduce human attention as a bottleneck, even if the underlying browser work were automated.

The final run expands to 20 companies, with Chipotle, J.P. Morgan and Goldman Sachs among the examples named as the display scrolls. The same code runs across the websites. The narration calls this a run over the “same three sources,” after naming four in the preceding demonstration, so the final per-company source count is unclear. The supported change is the expansion from researching one company to applying the shared workflow across twenty.

What comes back from that larger run? The presentation moves from the browser sessions to results, including SEC filing data. Browserbase’s hosted browser handles access and CAPTCHA friction, while Stagehand navigates and extracts without someone watching each session. The presenters describe uninterrupted execution in this demonstration; they do not supply a broader task-success rate. The result illustrates the combined mechanism: repeat the research task across different sites and collect its outputs.

There is a useful limit in the closing reaction to those SEC results: “I’m not a finance guy.” Collecting financial information and interpreting its significance are separate jobs. The demonstration reaches the first of those outcomes. The closing invitation offers a reusable SEC template and a copyable setup prompt for an agent, turning the example into a starting point for building a research workflow. The practical next step is to adapt the collection task and desired output to the financial question the application actually needs to answer.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:06 · section reference included

Resources

Transcript download
Full transcript

Loading transcript…