AI Engineer World's Fair 2024
GraphRAG: The Marriage of Knowledge Graphs and RAG
Read the talk
GraphRAG: Combining Knowledge Graphs and Vector Search for Better Retrieval
Emil Eifrem explains how knowledge graphs add explicit relationships, richer context, and interpretability to retrieval-augmented generation, while acknowledging the practical difficulty of constructing those graphs.
From a talk by Emil Eifrem
At a glance
Ideas worth remembering
GraphRAG means using a knowledge graph on the retrieval path, potentially alongside vector search rather than instead of it. 4:19
A practical retrieval pipeline starts with vector search, expands relevant nodes through graph relationships, optionally ranks the expanded set, and sends the selected context to the language model. 6:28
Reported accuracy gains come from specific examples cited in the talk; the transcript does not establish that the same improvements apply universally. 8:53
Explicit graph relationships can make retrieval behavior easier to visualize, debug, explain, and audit, while remaining complementary to vector representations. 11:00
The main implementation tradeoff is that GraphRAG becomes easier to develop against only after a useful knowledge graph exists, and constructing one from unstructured material can be difficult. 9:53
The Knowledge Graph Builder demonstration illustrates ingestion of documents and links, extraction into document chunks and concepts, and graph visualization, while leaving its chatbot functionality largely undemonstrated. 15:56
Why relationships matter in search
Emil Eifrem frames GraphRAG as part of a longer progression in search: moving from isolated data points and matching text toward systems that use relationships between pieces of information. Early web search engines relied on keyword-oriented methods, including inverted indexes and BM25-like approaches. As online content expanded, these systems could return thousands of matches, leaving users to identify the relevant results themselves. 0:14
The next step, in Eifrem’s account, was PageRank: a graph algorithm that uses the structure connecting web pages to surface important results earlier. He then describes the introduction of knowledge graphs, which represent concepts found within documents rather than treating the documents and links alone as the complete search substrate. 1:55
A knowledge graph consists of nodes, relationships, and key-value properties attached to both. Eifrem points to search results that combine unstructured descriptive text with structured facts as an illustration of how a graph-backed representation can connect different forms of information. His central argument is that generative applications can similarly benefit when language models retrieve information through explicit relationships. 3:01
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The basic GraphRAG retrieval pattern
Eifrem defines GraphRAG as retrieval-augmented generation that uses a knowledge graph somewhere along the retrieval path. This definition does not require replacing other retrieval methods: graph retrieval can be combined with vector search, allowing similarity-based discovery and relationship-based expansion to contribute to the same answer. 4:19
His example is a customer-support assistant for a company that builds Wi-Fi routers. Support articles become graph nodes with their text stored as properties. Each article can also connect to a specific product, a hierarchy of related products, and the support engineer who wrote it. The graph therefore preserves relationships that would otherwise remain implicit in the article text or external application logic. 5:25
When a user asks about flashing yellow lights and dropped connections, the system first embeds the question and uses vector search to retrieve relevant articles. It then traverses the graph to gather additional context, such as articles concerning related products or signals indicating that a particular engineer’s material should rank more highly. The expanded context and the original question are passed to the language model to generate an answer. 6:28
Graph expansion introduces its own selection problem: following relationships may produce thousands of nodes. Eifrem therefore describes an optional ranking stage that selects the top K results according to graph structure, potentially using PageRank, before sending the final context to the model. The resulting pattern is vector retrieval, graph traversal, optional graph-aware ranking, and language-model generation. 6:28
Question about a support issue
Vector retrieval seeds graph expansion, with optional ranking before generation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What graphs can add to answer quality
The first benefit Eifrem emphasizes is accuracy. He describes research and examples in which combining graph-based retrieval with vector search improved response quality relative to a vector-only or baseline retrieval approach. The proposed mechanism is straightforward: relationships help the retrieval system assemble relevant context that similarity search alone might not return. 6:28
He cites a data.world example involving approximately 43 questions and describes an average response-quality improvement of roughly threefold when a knowledge graph was combined with vector search. He also references work from LinkedIn that he recalls as showing an accuracy increase of approximately 75% or 77%. These figures are presented as his descriptions of individual examples, not as a universal performance guarantee. 8:53
Eifrem also describes Microsoft’s argument that GraphRAG can address an additional class of questions that baseline vector search or baseline RAG cannot answer. He does not specify those question types in detail, so the supported conclusion is limited: graph-based retrieval may change not only answer quality but also the kinds of questions a retrieval system can support. 9:53
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Explicit structure improves development and oversight
Eifrem’s second claimed benefit is easier application development, but he attaches an important condition: the advantage applies once a usable knowledge graph already exists. Teams must first learn how to construct that graph, and he acknowledges that this initial step can be difficult. The development benefit should therefore be understood as conditional rather than as a promise that GraphRAG is always simpler end to end. 9:53
He describes an unnamed fintech company that moved an existing application from a vector database to Neo4j. In his account, the company found that related actions could be retrieved by following relationships, embeddings could be calculated at the database level, and a visual representation of the application’s data helped identify bugs. The example illustrates how explicit graph structure can make application behavior easier for developers to inspect. 11:00
To explain why, Eifrem contrasts the statement that apples and oranges are both fruit when represented in a graph versus in vector space. A graph can explicitly represent the relationships between apple, orange, and fruit, whereas the internal meaning of a vector representation is not readily interpretable by a person. He emphasizes that graph and vector representations are complementary, while arguing that graphs offer a clearer working representation during application development. 11:00
The same explicitness also supports operational concerns beyond programming. Eifrem connects visible relationships and inspectable data structures with explainability, auditability, and governance, especially when an organization needs to understand why a production system produced a particular result. These are presented as benefits of representation and traceability, not as proof that every GraphRAG system automatically satisfies a particular governance standard. 13:19
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Building the knowledge graph is the central implementation challenge
Eifrem divides graph-construction inputs into three categories: structured data, unstructured data, and mixed data consisting of structured records with some long-form text fields. His structured examples include systems such as Snowflake, Postgres, MySQL, and Oracle; his unstructured examples include PDF files and raw webpage text. He argues that mixed data accounts for many enterprise production use cases. 13:19
In his assessment, converting structured data into a property graph is comparatively straightforward, while extracting a useful graph from unstructured content is difficult both conceptually and because tooling has historically been immature. He briefly identifies lexical graphs and domain graphs as relevant distinctions but does not explain them, so the transcript does not support a more detailed account of their differences. 14:32
To demonstrate an approach to unstructured inputs, Eifrem introduces the Knowledge Graph Builder. He says it can ingest PDF files, YouTube links, Wikipedia links, and cloud-service buckets, then extract information and construct a graph. His example inputs include Andrew Ng’s newsletter The Batch, the Wikipedia page for OpenAI, and a Latent Space podcast episode he identifies as Four Wars. 15:42
The live demonstration initially struggles with connectivity and rendering before displaying a graph. Eifrem shows a document connected to chunks, describes expanding those chunks into extracted concepts such as machine learning, and mentions that embeddings are included. He also notes that the tool contains a chatbot with introspectable results, but time constraints prevent him from demonstrating that functionality in detail. 15:56
PDFs, links, and cloud buckets
Uploaded source material becomes a graph linking documents, chunks, and extracted concepts.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:00
[on-hold music] I've basically dedicated my professional life towards getting developers to be able to build better applications and build applications better by leveraging not
- 0:25
just individual data points kind of retrieved at once, like one at a time, or summed up or grouped calculated averages, but individual data points connected by relationships. Right? And today I'm gonna talk about that applied in the world of LLMs and GenAI.
- 0:44
So before I do that, though, I'm gonna take a little bit of a detour. I'm gonna talk about search, the evolution of search. Everyone here in this room knows that the vast majority of web searches today are handled with Google.
- 0:54
But some of you know that it didn't start that way. It started this way. Who here recognizes this webpage? Right, yeah. Who here recognizes AltaVista as a name? Like, a few, a few people, right?
- 1:05
Um, back in the mid-'90s, there was dozens of web search company. Dozens plural, like 30, 40, 50 web search companies, and they all used basically the same technology. They use, used keyword-based text search, inverted index-type search, BM25-like, for those of you who know what that means.
- 1:22
And it worked really, really well until it didn't. And the AltaVista effect kicked in which was the notion that you search for something, you got a th- thousand or thousands of hits back, and you had to look through page after page until you found the result that was relevant to you.
- 1:40
The AltaVista effect. You got too much back from the internet. That wasn't a problem in the beginning because m- most of the things you searched for when I went on, onto the internet in the beginning got zero results back 'cause there was no content about that on the internet, right?
- 1:55
But the AltaVista effect, too many search results, was solved by Google. This is Google's press release mid, you know, mid-2000. They talk about a billion URLs they've indexed, right?
- 2:06
But they also talk about the, the technology that they use behind the scenes, the technology called PageRank that delivers the most important search results really early on. In fact, the first, the top 10 blue links on that first page.
- 2:20
Right? That technology, PageRank, is actually a graph algorithm which is ... Actually, it's called Eigenvector centrality. And the innovation that Google did was applying that to the scale of the internet and the scale of the web.
- 2:35
Right? PageRank. That ushered in and created honestly the most valuable company on the planet for, uh, quite, quite some while, the PageRank era. Right? That lasted for about a decade, about a dozen years until in 2012 Google wrote this blog post, which is an amazing blog post, introducing the Knowledge Graph, "Things, not strings," where they said, "You
- 3:00
know what, guys? We've done an upgrade on the backend of our search technology, the biggest one since we invented PageRank, where we're mo- moving away from not just storing the text and the links between the documents, but also the concepts embedded in those documents.
- 3:17
Things, not just strings." And we all know what the Knowledge Graph looks like visually. When you search for something on, on, on Google today, Moscone Center just around the, the corner from here, you're gonna get this little panel right on the right-hand side.
- 3:32
If you look at that panel, it has a combination of unstructured text, in this case from Wikipedia, with structured text. It has the address, the owner of the Moscone building, you know, that kind of stuff.
- 3:46
This thing is backed on the backend by the data structure looking like this, right? It has these concept, the rings that we call nodes that are connected to other nodes through relationships.
- 3:58
And the, both the nodes and the relationships have key value properties. You can attach two, three, a thousand, 10,000 on both the nodes and very importantly also on the relationships.
- 4:09
This is a knowledge graph, and that was the next decade or so, 12 years, of Google's dominance until a few months ago. A few months ago at Google I/O, they took the next step ushered in by the AI Engineers conference a year ago.
- 4:27
Well, not quite, but of course the entire craze around GenAI. And this was one of the example that they did, the classic travel itinerary. They helped me plan out this, this travel.
- 4:37
Everyone here in this room knows that this is backed by an LLM, and it is backed by an LLM in combination with this knowledge graph data structure, GraphRAG. This is ushering in the next era of web search, the GraphRAG era.
- 4:54
What I'm gonna talk to you about today is how can you use ... Well, first of all, should you, and if so, how can you use GraphRAG for your own RAG-based applications?
- 5:06
So what is GraphRAG, right? It is very, very simple. GraphRAG is RAG where on the retrieval path you use a knowledge graph. Very, very simple. It doesn't say you only use a knowledge graph, but you use a, a knowledge graph maybe in combination with other technologies like vector search.
- 5:25
So let's take the classic example of a customer service bot, right? And let's say that you are working at a company that is building Wi-Fi routers, for example, right?
- 5:36
And you have a bunch of support articles, right? And they've been stored in text files, right? And then you are tasked with building a bot that either is-- gives direct end users access to it or your own customer service agents, employees, like, access to this information.
- 5:53
And you know how to do this because you live in the LLM world, in the GenAI world, so you're gonna use RAG for this, right? And so you have that data.
- 6:01
It's text documents. You've added that text onto the properties of particular nodes, right? So you have a node per article, but then you've also said that, "You know what?
- 6:11
This article is about this particular Wi-Fi product," right? You have a relation to that Wi-Fi product, and that Wi-Fi product sits in a hierarchy of other Wi-Fi products, and it's written by this particular customer service engineer, you know, that kind of stuff.
- 6:26
And then the end user has a question. "Hey, my Wi-Fi lights are flashing yellow, and my connection drops. Like, what should I do?" Something like that. I think we all know how we do this.
- 6:37
We vectorize the search, right? We get a, some kind of vector embedding back. We use vector search to get the core documents. But here's where the GraphRAG part kicks in.
- 6:47
You'd get those core articles back which are linked to the nodes. Actually, the text is on the nodes. But then you use the graph to traverse from there and retrieve more context around it.
- 6:57
Maybe it's not just that particular articl- article for that particular Wi-Fi, but something else in that family. Maybe you use the fact that this particular engineer has very highly ranked content, and then you rank that higher, right?
- 7:11
You retrieve more context than what you get out of the ANN-based search from your, from your vector store, and you pass that on to the LLM along with the question.
- 7:21
You get an answer back, and you hand it to the user.
- 7:26
So the core pattern is actually really, really simple but really, really powerful, right? You start with doing a vector search. I think of this almost as a primary key.
- 7:36
It's of course not a primary but almost like a primary key lookup into the graph. You use that vector search. You get a, an initial set of nodes. Then you walk the graph, and you expand that and find relevant content based on the structure of the graph.
- 7:52
Then you take that, and you return it to the LLM, or optionally, maybe that gives you a thousand or ten thousand nodes back, and then you do what Google did.
- 8:00
You rank that. You get the top K based on the structure of the graph. Maybe you even use PageRank, right? You get that. You pass it on to the LLM.
- 8:09
Really, really simple but really, really powerful. And then there's a number more advanced patterns, but that's kinda the next, the next talk I, I'll do in a year. The, like, the, it's more sophisticated graph retrieval patterns, right?
- 8:21
But the core one, very, very simple. Okay.
- 8:27
So if that's what GraphRAG is, what are the benefits of GraphRAG? When should you use it? When should you not use it? The first and most stark benefit is accuracy.
- 8:38
It's directly correlated to the quality of the answer. There's been a ton of research articles about this in the last six months, something like that. I believe the first one was this one by data.world.
- 8:49
I just picked out three at, at, at random here that I, that I, that I like. This is the first one that I know of by data.world, which is a data cataloging company based on a knowledge graph, and they proved out across, I think, forty-three different questions that on average the response quality, the accuracy was three times
- 9:09
higher if they use a knowledge graph in combination with, with vector search.
- 9:15
I love this paper by LinkedIn. Uh, it's, uh, shows a very similar type. I think it's, like, seventy-five percent or seventy-seven percent increase in, in accuracy, um, but it also has a great architecture view, so you can take a, the, the QR code right there and look at that paper which combines various components and also the flow
- 9:34
through that that I thought it was just really pedagogical. Um, but by and large, it's showing the same thing, a little bit of different numbers but significantly higher accuracy when it used graph in combination with vector search.
- 9:47
And then Microsoft did a fantastic blog post and subsequently I think two academic papers, the blog post was in February of this year, where they also talk about the increased quality of response but also beyond that, "Hey, you know what?
- 10:02
GraphRAG enables us to answer another important class of do- of questions that we couldn't even do with vector search alone or baseline vector search," that's what they, "or baseline RAG alone."
- 10:15
So first benefit, higher quality response back. The second one is easier development, and this one is a little bit interesting 'cause there's an asterisk in there because what we hear very clearly from our users is that it's easier to build RAG applications with GraphRAG compared to baseline RAG, but we also hear it's, like, it's actually hard. [laughs]
- 10:37
And what's the nuance there? Well, the nuance is if you already have a knowledge graph up and running. So there's a learning curve where people need to learn, "How do I create the knowledge graph in the first place?"
- 10:47
Once you have that, it's a lot easier, but how do you create that knowledge graph, right? So let's put a little pin in that. If I rush through the next few slides quickly enough, I'm gonna show you hopefully a demo on, on, on, on that, but let's put a little pin in that.
- 11:02
So this is an example. This is from a, um, a very high growth stage fintech company that is very cutting edge in AI, and they started playing around with GraphRAG a few-- about six months ago.
- 11:16
And they took an existing application, and they said, "You know what? We are gonna port this from a vector database to Neo4j," and most of the operations yield a better result.
- 11:26
They can calculate the embeddings on a database level. Getting related actions is as simple as following the relationships between nodes. And this one I love. The cache, and the cache here is their application, they call it the cache, can be visualized.
- 11:40
This is an extremely valuable debugging tool, and in the parenthesis, I actually already fixed a couple of bugs just thanks to this.
- 11:48
Right? Amazing. Like, once you've been able to create that graph, it's a lot easier to build your RAG application. And why is that? Right? So let's talk a little bit about representation.
- 12:02
Let's say we have the phrase in there, "Apples and oranges are both fruit," and we wanna represent that in vector l- space and in graph space. In graph space, we already talked about this.
- 12:13
Apple is a fruit. Orange is a fruit. Pretty easy. That's the representation in graph space. In vector space, it looks like this. Maybe, or maybe this is something else, like we actually don't know.
- 12:27
Two different ways of representing that phrase, and then we can run similarity calculations in different ways using these both, both representations that I'm not gonna go through right now.
- 12:38
We can search in different ways. These are not competing ways of doing it. They're complementary ways of doing it, right? One is not better than the other, except I will make one statement, which is
- 12:51
when you sit down and you write your application, when you build your application, I'm actually gonna make the statement that one of them is superior. This vector space representation is completely opaque to a human being.
- 13:04
But the graph representation is very, very clear. It is explicit. It's deterministic. It's visual. You can see it. You can touch it as you build your applications. This is the, "I already fixed a couple of bugs thanks to this," just by porting it from a vector-only store to GraphRAG, they were able to see and work with the
- 13:24
data, and that is really fricking powerful. That shows up in development time as you're building your applications. It's also showing up for our friends in IT who worry about things maybe that is not directly related to building the application, which is explainability, which is auditability, which is governance.
- 13:47
That explicit data structure has knock-on effects over there that are really, really powerful once you're up and running in production and you need to build- be able to explain why something happened.
- 14:02
So higher accuracy, better answers, easier to build once you're through the hump of creating the knowledge graph, and then increased explainability and governance for IT and the business, right?
- 14:14
Those are the three things. So how do you get started with, with GraphRAG? Well, I've talked a lot about this already. Like, how do you create the knowledge graph in the first place?
- 14:24
So a little bit of nuance here. So basically, there are three types of data out in the world that I care about when I think about knowledge graph creation.
- 14:32
The first one is structured data. So this is your data in your Snowflake or something like that, or Postgres, right?
- 14:40
The other one is unstructured data, PDF files, raw text from a webpage. And the other one, the third one is mixed. People tend to call this semi-structured, but it's not.
- 14:49
Hit me up afterwards and I'll tell you why it's not. But basically what this one is, is structured data where some of the fields are long-form text, right? Basically, we're great in the first bucket in the graph world.
- 15:02
It's very easy to go from Snowflake or Postgres or MySQL or Oracle into a property graph model.
- 15:10
The unstructured one is really fricking hard, right? It's hard to do in theory. It's also had immature tooling for a long time. The middle one is actually where the majority of at least enterprise production use cases are in the real world.
- 15:29
So, man, two and a half minutes. This is rough. Um, there are two types of graphs, and I'm not gonna talk about them. I wanna talk about them. Lexical graphs and domain graphs is actually really relevant, but I really wanna get to this demo.
- 15:42
So I've talked about creating graphs with unstructured information. So we just built this new tool that we launched just a few weeks ago called the Knowledge Graph Builder, and you see it here.
- 15:54
I... Can, can you see the screen okay? Yeah. So basically here you can drag and drop your PDF files. You can put in YouTube links, Wikipedia links. You can point it to your kind of [REDACTED:username] service bucket, right?
- 16:06
And it's gonna extract the data from there and create the graph. So here I added a few things. I added, um, a PDF of Andrew Ng's newsletter, The Batch.
- 16:17
I added the Wikipedia page for OpenAI, and I added the YouTube from swyx and Alessio's, you know, the Four Wars, uh, Latent Space podcast. So I added all that, and I uploaded it into this Knowledge Graph Builder, and when I do that, it creates if-- Let's see here.
- 16:36
I knew the Ethernet connection was gonna do it. It auto-magically created a little knowledge graph if it renders. Wait for it. I- it says one minute here. [laughs] So it better render pretty soon.
- 16:53
All right. Let me do this again. Please work. [laughs]
- 17:00
Oh, no. [laughs] Yeah. Oh, man, my Wi-Fi isn't... Oh. [laughs] Oh, crap. Oh, no. [laughs]
- 17:11
And it's ticking down. All right. Wait for it. Wait for it. All right.
- 17:20
You can do it. Didn't do it. And I was, like, trying to keep it alive in the, in the thing too. All right. Okay. Let's see. I think we are here, and then it says, "Show me a graph," and it's not gonna show me the graph.
- 17:38
Oh, yeah, it will. Come on. I think we can do it. All right. Yes. [clapping] [cheering]
- 17:45
So what we have here, check this shit out. I would love to sit here and just drink in your applause, but we need to look at this data. So check this out.
- 17:54
This is the document, the Four Wars document. Here are the various chunks, and then you can take a chunk, and you can expand that. This I put in the, the, the embedding, and you can-- I'll zoom out here, and you can see that it takes the, the logical concept elements out of that chunk, right?
- 18:14
Machine learning. They talk about something that is developed in a similar fashion. I don't even know. There's some company there, right? And you get that entire graph of all this information.
- 18:26
On top of that, I really don't have time to show it, but there's also-- I really don't have time to show it. [laughs] There's a chatbot in here that you can use, and you can introspect the result that gets back.
- 18:37
I'll-- One more second. Take out your phones. If you think this looks cool, take a photo of this QR code, and you're gonna have an amazing landing page where you have access to all of this information.
- 18:49
You can get up and running yourself. Thank you for the additional minute. Thank you, Emil. Thanks, everyone, for paying attention. [clapping] [upbeat music]