Why Datasite Stopped Building Its Own Parsing - Adit Abraham, Reducto & Raj Bakhru, Datasite

Read the talk

Why Datasite Stopped Building Its Own Document Parser

When BlueFlame traced a wrong answer for a private equity customer back to a misread pie chart, its engineers saw how much could go wrong before a model answered a question. Datasite later credited Reducto’s parsing with a 22% improvement in recall on its internal deal-room benchmark.

From a talk by Adit Abraham and Raj Bakhru

At a glance

Ideas worth remembering

  • Trace a wrong answer back to the source figure and its extracted representation. In BlueFlame’s pie-chart case, the error was the association between a legend color and a slice.

  • Specify both sides of a parser integration: where sensitive files can be processed and what extracted output the retrieval system will consume. Size ingestion for the bursts customers actually create.

  • Evaluate agent models on completed tasks. Repeated tool calls can change runtime and cost enough to complicate a choice based on price per call.

  • Test extraction changes with a fixed deal room and known-answer questions. Running the same files through a general-purpose project gives a custom system a meaningful comparison.

From building trading systems to working on the deals themselves

Adit Abraham, co-founder and CEO of Reducto, opens with a figure he wants Raj Bakhru to confirm. Roughly 60% of transactions worth more than $100 million go through Datasite, where Bakhru is global CTO. Bakhru confirms it. Abraham’s interest is how the engineering decisions look from two positions Bakhru has held. He helped found BlueFlame, an early vertical AI company serving mergers and acquisitions, then joined the company that already hosted the documents for much of that market.

Bakhru came to M&A by way of software development. At Goldman, Highbridge and Kepos Capital, he built research and trading systems for quant funds. He then started a cybersecurity firm and sold it to a private equity-backed company, where he ended up running M&A and corporate strategy. By the time he started BlueFlame, he had experience both building systems for finance professionals and doing transaction work himself. The founders saw an opportunity in the volume of unstructured information people had to interpret to complete a deal.

BlueFlame spent roughly a year to a year and a half building before Datasite approached it about an acquisition. Bakhru and his co-founders were financing much of the work themselves. Building their own parsing, document processing and ingestion pipeline was one way to conserve cash. It worked well enough to put a product in customers’ hands. Those customers would give the engineers much more demanding examples than the ones that had convinced them the pipeline worked. BlueFlame had already started working with Reducto while it was still independent.

0:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

Putting question answering inside 55,000 deal rooms a year

BlueFlame had about 50 people when Datasite acquired it. Its first assignment was to bring AI into Datasite’s virtual data rooms, or VDRs. When a company is for sale, bankers use a room to share the transaction documents with prospective buyers, including private equity firms. Diligence providers also use those materials. A user submitting a question inside a room wants an answer grounded in the documents for that particular sale.

Bakhru puts the scale at billions of pages of content, more than 500,000 monthly active users and about 55,000 data rooms a year. Users were asking for AI features regularly, and the board wanted them too. To answer questions across those rooms, the team needed to parse the files, ingest the extracted content into a vector database and build its graph-based retrieval-augmented generation system, or GraphRAG, on top. That sequence made extraction quality an issue for everyone working downstream. A query could reach the right document and still yield the wrong answer if the parser had misread the relevant figure.

There was also a constraint on where the processing could happen. Datasite had historically kept the contents of its data rooms inside its Azure environment. The files held material nonpublic information about transactions. Clients had regulatory obligations that carried through to Datasite, and Datasite had its own certifications, including ISO certifications, to maintain. Bakhru’s team spent time with Abraham’s working out how to use Reducto while keeping the content protected. Choosing a parser on quality, speed, recall and cost was one decision. Establishing where it could operate on live deal documents was another.

2:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:12 · section reference included

Tracing a private equity firm’s wrong answer back to a pie chart

BlueFlame’s data science team benchmarked the extraction and ingestion tooling the founders had built. Its most instructive debugging material came from private equity customers who sent back questions the RAG system had answered incorrectly. The support case arrived as a bad answer. To understand it, someone had to work backward to the source and determine what information the answering system had actually received.

In one case the source was a pie chart. Reading it required associating a shade of blue in the legend with the corresponding slice. Even using what the team considered the best available vision model, the model read that association incorrectly. Bakhru says the same model would have read the same data correctly as a bar chart. The financial question and underlying numbers could stay the same while the representation changed which fact the system extracted. Improving the prose of the final answer would do little for a value already attached to the wrong category.

Finance workbooks produced a different class of support case. Merged cells made it difficult for a model to interpret how a sheet fit together. Bakhru points out that the workbooks can be hard even for a person to understand. And making sense of the layout in one workbook addressed only part of the job. The team also encountered binary Excel workbooks, macro-enabled files and template files. Each was another form in which a client could reasonably expect to submit financial information.

4:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:41 · section reference included

Drawing the supplier boundary at the output of extraction

For Bakhru, the cost of owning the parser became apparent in the support queue. Someone had to investigate the chart, work out which model read that type of visual reliably and then take the next case involving a different chart or spreadsheet. The original decision to build had saved money when cash was scarce. Maintaining that decision committed a limited group of engineers and data scientists to an ongoing document-format problem.

BlueFlame chose to hand much of that work to Reducto. The handoff had a concrete interface. Reducto would supply extracted content, and BlueFlame would use that output in its own GraphRAG system. Its people could work on how to retrieve information across deal documents, what the client could do through the interface and which diligence tasks an agent should perform. Bakhru wanted the engineers who understood clients’ questions to spend their time improving those parts of the product. A data scientist investigating the next spreadsheet exception was someone unavailable to improve retrieval for a whole room.

6:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:11 · section reference included

A $5 million investment request turned into an acquisition offer

Abraham returns to the independent company. BlueFlame had entered vertical AI early, when the models available to do this work were less capable. What had the founders originally hoped to build? Bakhru says they expected to be at it for a long time. They had sold their cybersecurity startup about a year in. This time they wanted to see how far they could take a company themselves.

Being second-time founders changed who they could recruit and how they could finance the work. They brought in people they had worked with before, knew one another’s families and could disagree vigorously without damaging a friendship. They could also afford to bootstrap for a while. Bakhru connects that freedom to some decisions he considers excessively cost-sensitive in retrospect. When the money for a specialist tool comes out of the founders’ own pockets, building it can look particularly attractive.

BlueFlame was about to close its first venture round. The founders had met investors on Sand Hill Road and in New York, signed a term sheet and were about a week from final signatures. There was still room in the round, so they approached Datasite about investing $5 million. They demonstrated the product at an office meeting and went to dinner that evening. Over dinner, Datasite told them it wanted to buy the whole company. The founders suddenly had a decision quite different from how to fill the remainder of a financing round.

7:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:17 · section reference included

Datasite offered two kinds of proprietary deal data

Bakhru asks Abraham to put the decision in its market context, roughly a year and a half before their conversation. Agentic coding had yet to reach the level of adoption they were now seeing. Some vertical AI competitors had already raised hundreds of millions of dollars. The founders believed that staying independent and competitive would require raising money on a similar scale, with substantial dilution. They also saw signs that the frontier model providers were developing offerings for financial services. The question was what BlueFlame could offer a finance user as those providers moved into its market.

Datasite’s rooms offered one answer. They held proprietary documents that bankers shared with buyers and advisers during a sale. Bakhru believed that joining Datasite would give BlueFlame access to a source of deal data that competing AI providers could not offer at the time. To put that advantage to use, the team had to be able to analyze a room’s documents inside Datasite’s protected environment.

A second source was Grata, a Datasite company with a database covering about 20 million private companies, including transaction and contact data. Bakhru gives the example of a private equity firm seeking add-on acquisitions for a company already in its portfolio. That is a sourcing or origination question. Grata can help someone identify companies to consider, while a data room holds the evidence buyers and advisers examine once a particular company is in a sale process. BlueFlame saw value in bringing both kinds of information into its product.

9:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:52 · section reference included

From questions about files to diligence decks and committee memos

Over the following year and a half, the companies brought their engineering teams, data and products closer together. Bakhru says the team benchmarked what happened as it brought the data closer to the models. It saw lower token costs, shorter runtimes and higher quality. He gives the direction of all three changes without specifying magnitudes or an implementation detail that would isolate the cause of each.

Demand grew along two dimensions. Bakhru estimates that Datasite’s top line grew about 40% over that period and says the volume of incoming content was growing rapidly. At the same time, customers were asking the system to do more with the documents already in a room. An individual question might require retrieving a fact. A diligence agent could iterate over documents, perform a more involved analysis and produce a PowerPoint deck or an investment committee memo. Extraction now supplied evidence for a deliverable that other people would read to make a decision.

11:22
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:22 · section reference included

How Bakhru allocates 250 engineers across five initiatives

Abraham asks how Bakhru now makes build-versus-buy decisions for a global workforce as model capabilities improve. Datasite spends substantially with vendors, Bakhru says, and does so deliberately. A supplier dedicated to a problem can keep working on it every day. For Datasite, assigning people to the same problem means recruiting scarce engineers and carrying the ensuing support burden alongside the product work it hired them to do.

AI coding tools have also affected whom Bakhru wants to recruit. He describes three groups. Experienced engineers who understand engineering principles and have a strong mental model of a codebase get considerable leverage from agentic coding. He has seen a more modest benefit among engineers in the middle. Junior engineers gain in a different way. Where missing documentation once left them trying to work out how the code fit together, they can ask a model to generate an explanation as they go. That observation underlies his preference for hiring strong senior engineers alongside junior people the team can develop.

He puts the allocation decision in financial terms. Using roughly a billion dollars as the order of magnitude for Datasite’s annual revenue, he estimates that the board’s ambition of 20% annual growth requires about $200 million in incremental revenue each year. The organization has about 250 engineers and five major revenue-generating initiatives. The work includes building products, consolidating existing tools and developing new commercial or monetization models. Bakhru wants as many engineers as possible on those five initiatives and as few as possible maintaining the underlying layers a vendor can operate. That is the company-wide version of his earlier decision about who should handle an unreadable spreadsheet.

12:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:44 · section reference included

A cheaper agent model keeps calling tools

Before opening questions to the room, Abraham asks where Bakhru still encounters friction. Buying parsing removed much of the work of finding the right vision model for a given chart or document. The team still has to make model choices for its agents. Bakhru describes one candidate that repeatedly issues tool calls in circumstances where another model avoids the repetition. The first is sufficiently cheaper that the team continues trying to make it behave as intended.

The comparison illustrates what Datasite has to benchmark. The price of a model call is only one input to the cost of finishing a diligence task. Repeated calls also consume time, and a fast run is of limited value if it produces a poor answer. Bakhru names three criteria: cost, speed and quality. For finance users seeking the best answer available, he gives quality first priority and works on cost and speed subject to that requirement.

15:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:54 · section reference included

The live ingestion path from a banker’s upload to a Python query

An audience member asks how Datasite and Reducto actually integrate. Bakhru says Reducto operates inside Datasite’s environment against its content. Alongside the billions of pages already on the platform, the system has to accommodate abrupt bursts. A banker can drop up to a terabyte of data into it at short notice. The annual count of rooms gives a sense of total demand, but this single-upload example explains the ingestion problem the integration has to handle.

Bakhru follows the data in order. Datasite’s ingestion pipeline takes the incoming files and passes them through Reducto. Datasite receives the extracted output and sends it to turbopuffer, its vector database. It builds GraphRAG on top of that and executes its GraphRAG queries in Python. The deployment answers two separate integration questions. The processing happens within Datasite’s environment, and Datasite retains control of the retrieval and query code that uses the parser’s output.

17:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:03 · section reference included

VDR Bench measures what better parsing recovers

A sell-side Datasite customer describes a file system containing many transactions. Their firm wants agents to use the material and wants to anonymize the material nonpublic information before passing it to a tool such as Reducto. Facing its own build-or-buy decision, the customer asks how Datasite constructed its evaluation set. Bakhru answers the evaluation question. Anonymization was the customer’s proposed requirement, and he gives no anonymization procedure.

Of Datasite’s several custom evaluations, the one most relevant to parsing is VDR Bench. The team takes a representative virtual data room containing hundreds of documents and asks a few hundred questions for which it has known answers. It reruns the evaluation after changes to almost any part of the system. Keeping the room, questions and expected answers available gives the team a way to judge a change at the level the client experiences it.

Bakhru’s benchmark for the product was what a customer could do by loading the same files into Claude Projects or OpenAI Projects and asking the same questions. He says Datasite beat that comparison and credits much of its advantage to how it extracted the content. Within Datasite’s own system, the data science team attributed a 22% improvement in recall to Reducto parsing. Bakhru gives neither the starting recall nor the scoring formula. VDR Bench was internal at the time. The team was considering releasing it and working with OpenAI on a paper.

Bakhru’s conclusion is that, for Datasite’s deal-room questions, most of the work of getting a good answer happens during extraction. The pie chart shows why. Once the parser assigns a slice’s value to the wrong category, the answering model can faithfully report the extracted figure and still give the investor the wrong answer. VDR Bench lets the team test whether a change to extraction helps it recover the right information across hundreds of questions.

17:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:46 · section reference included

Transcript download
Full transcript

Loading transcript…