AI Engineer World's Fair 2024
Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications
Loading slides…
Read the talk
Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications
Build a retail graph, search product descriptions, personalize results through purchase paths, and use graph embeddings to supply recommendations for a seasonal email.
From a talk by Andreas Kollegger, Zach Blumenfeld, Michael Hunger and Tomasz
Before you start: You should be comfortable reading Python notebooks and basic database queries; the article introduces the graph and vector-search concepts used in the workshop.
Prepare the notebook and graph database
What needs to be running before a Python notebook can retrieve real retail data from a graph? The workshop starts with two pieces: a blank Neo4j Graph Data Science sandbox and an editable notebook. Zach leads the implementation, with Andreas, Michael Hunger and Tomasz helping participants through the alternating slides and notebook exercises.
- Create a sandbox by choosing For Data Science, then the blank Graph Data Science option. Keep its generated connection credentials.
- Open the GenAI Workshop notebook from the workshop repository. In Colab, use File → Save a Copy in Drive so you can edit it. VS Code or a local notebook environment is also suitable.
- Run the dependency installation cell early; loading the packages takes time.
These are the recorded setup steps. The maintained workshop materials have since moved to AuraDS and reorganized the notebooks, so their current setup differs from the recording.
If you accidentally create the regular blank sandbox, the workshop’s correction is to terminate that instance and create the GDS variant. The regular instance should still work, but the GDS sandbox has more memory for the heavier operations that come later.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Find an item, personalize it, then suggest what goes with it
Retrieval-augmented generation inserts a data lookup between a user’s request and a language model’s response. That lookup supplies domain context and traceable source material. Here, GraphRAG combines vector search, graph traversal and graph data science: descriptions identify relevant products, purchase relationships personalize them, and graph embeddings provide additional recommendations.
Consider a customer looking for a halter neck top. First, the application needs relevant items from the store’s catalog. Then it needs to choose among those items for this particular customer. Finally, it may suggest pants, a hat or accessories to pair with the top. The workshop uses H&M competition data to build that pipeline and lets the language model turn its results into an email. The same separation between retrieval and composition can serve support agents, internal search and other customer experiences.
The repository also contains workshop slides, although Zach notes that the PDF available during the session differs from his live deck. The finished interface is a small Gradio app accepting customer information, interests and a time of year. OpenAI supplies the language model, LangChain orchestrates the vector stores and generation chain, and Colab hosts the notebook; other model providers could fill the generation role. The implementation proceeds from graph construction to vector search, purchase-based personalization, graph embeddings and nearest neighbors, and finally email generation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build the customer–article–product graph
The data model separates a product from its sellable variants. A customer purchases an Article, and that article is a variant of a Product. A jeans style is the product; its colors, patterns or sizes distinguish variants. Articles also connect to departments, though the workshop makes little use of department nodes.
| Entity | Relevant properties |
|---|---|
| Customer | Unique customer ID |
| Article | Color and design pattern |
| Product | Name, description, type and garment group |
| Department | Department association |
This separation lets purchase behavior remain specific to articles while semantic search operates on product descriptions.
After installing packages and importing the notebook’s libraries, copy your sandbox’s Bolt connection URL into the Neo4j URI setting, supply its password, and retain neo4j as the username. The generation model is GPT-4o. Use your own OpenAI credentials; the key offered in the session was temporary. The notebook also offers a separate environment-file route for configuration. The Graph Data Science client connects with the URI and credentials, and the following debug cell checks that connection.
The preparation notebook supplies five CSVs: department, product, article, customer and transaction. The first four describe entities; transactions supply customer-to-article purchase relationships. The loader creates uniqueness constraints, loads nodes, then loads relationships. Its generated Cypher uses UNWIND, MERGE and SET: expand input rows, identify or create the corresponding graph objects, and populate their properties. Zach points to GraphAcademy for a deeper treatment of graph modeling and ingestion.
Neo4j Browser provides an immediate inspection surface. You can sign in with the database username and password instead of single sign-on. The database panel lists node labels, relationship types and properties; selecting the Article label runs a MATCH query displaying a small sample of nodes.
Two finishing steps prepare products for retrieval and output. Because the dataset lacks product links, the loader creates placeholder URLs; links generated from them are not real shopping destinations. It also combines product name, type, garment group and description into one text property. That combined field becomes the input to text embedding.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate embeddings before creating the index
An embedding compresses information into a numeric vector whose geometry preserves useful similarities. For text, that similarity concerns meaning. For a graph, it can concern a node’s position or surrounding structure. Cosine similarity and Euclidean distance provide ways to compare vectors; later, the same broad idea will represent purchasing relationships rather than descriptions.
Neo4j has several index families: range indexes for equality and range comparisons, point indexes for spatial data, text indexes, and Lucene-backed full-text indexes. The workshop uses a vector index backed by HNSW, hierarchical navigable small world, for approximate nearest-neighbor search. A query such as halter neck top is embedded into the same space as product text so the index can locate semantically related products. Browser visualization remains available through the sandbox’s Open action, but the notebook first completes indexing.
The implementation has three distinct stages:
- Select products with detailed descriptions, avoiding incomplete records.
- Use
genai.vector.encodeBatchto request embeddings and store the returned arrays as a text-embedding property on those products. - Create a vector index over that property, configure cosine similarity, and wait for the index to become online.
The recording uses OpenAI’s Ada embedding family. Azure OpenAI, Vertex AI and AWS Bedrock are mentioned as alternatives, and embeddings generated outside Neo4j can also be ingested. Writing an embedding property does not create an index: the first operation stores the representation; the second makes it efficiently searchable.
The first search prompt is denim jeans. Encoding it produces a query vector; db.index.vector.queryNodes then retrieves product nodes and their similarity scores. With the index name and encoded vector supplied as parameters, the retrieval core is:
cypher
CALL db.index.vector.queryNodes($index_name, 10, $query_vector)
YIELD node AS product, score
RETURN product.text AS text, score
ORDER BY score DESC;
The workshop also returns product identifiers alongside the text. The resulting descriptions are jeans and denim products: a useful baseline before customer-specific evidence enters the query.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Wrap retrieval without coupling it to one storage layout
An attendee encounters an unknown-property warning, but no diagnosis is established in the discussion. A separate question clarifies the index structure: the HNSW graph is internal to each vector index, distinct from the customer–article–product graph. The recorded implementation uses Lucene underneath that index.
Existing embeddings can be imported with dimensions matching the index. Zach points to a commented LangChain example for external embedding generation, while noting that he has not run that block recently. Keeping vector retrieval inside Neo4j makes it convenient to continue directly into graph traversal, but it is not required: an orchestration layer can query an external vector store and then use Neo4j for connected-data retrieval.
The graph becomes useful when the application needs structured personalization or graph-derived embeddings beyond plain text search. It can also persist precomputed nearest-neighbor relationships for later lookup. Zach cites index-free adjacency, expressive multi-hop patterns and a flexible schema as reasons to use a graph database. His answer that scalar quantization is not yet available belongs to the recording: the vector-index manual documents quantization configuration introduced in Neo4j 5.23.
LangChain’s Neo4jVector now wraps the existing index using its name, the database credentials and an OpenAI embedding object. The workshop uses 1,536-dimensional text embeddings. This wrapper returns documents containing product descriptions, URLs and metadata, which can also be displayed as a dataframe. Searching for denim jeans reproduces the earlier retrieval; changing the prompt to oversized sweaters produces matching sweater descriptions. The wrapper prepares the same retrieval operation for the eventual generation chain.
The scaling discussion does not establish a fixed capacity limit or automatic-sharding guarantee. Asked about billions of documents, Zach describes indexes instantiated over selected properties, metadata-based federation being explored, and an external vector store as another architecture for very large collections. The application can replace the vector-retrieval call while retaining its Neo4j traversal stage.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Derive a peer group from shared purchases
Vector search finds relevant descriptions. Graph pattern matching then follows the entities associated with those descriptions. In this dataset, customers who purchase similar articles provide evidence about what one another might want next—the familiar collaborative filtering pattern expressed as a traversal.
Browser makes the relationships concrete. Its schema view shows customers, articles, products and departments; an additional inferred relationship visible in the presenter’s database has not yet been created in the participants’ databases. A sample query follows products to their article variants and then to purchasing customers. In the displayed graph, customers are orange, articles blue and products red. Purchase relationships carry dates and prices, while product nodes expose the assembled text and stored OpenAI embeddings.
The next query fixes a customer ID as a reusable parameter and retrieves that customer’s purchase history, including product names, types and descriptions. From there, the recommendation traversal proceeds in order:
- Find the customer’s latest purchase date.
- Select the articles purchased on that date.
- Find other customers who purchased those articles.
- Follow those customers’ purchases to additional articles.
- Return candidates the original customer has not already bought.
Zach illustrates the idea with ABK: if they buy similar things, ABK’s other purchases may reveal something Zach would like.
The resulting common purchase score counts matching co-purchase paths. It measures how often a candidate appears through the selected peer group; it is not itself a probability of purchase. Sweaters and T-shirts appear among the ranked candidates. The peer group is derived from shared purchases, rather than assigned as a demographic category: start at a customer, cross to a purchased article, cross back to its other buyers, then follow their other purchases. More matching paths give the candidate more support under this rule.
The traversal can extend farther, and its time scope is adjustable. The demonstration selects the maximum purchase date. A condition on t_dat, the purchase-date field, could instead select the last week or the last two weeks. That choice determines which behavior defines the customer’s current neighborhood.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rerank semantic matches without losing cold-start results
A custom retrieval_query on Neo4jVector appends the purchase-pattern logic after vector search. The application passes a customer ID along with the query, finds that customer’s peer group, and scores how strongly each retrieved product appears in it. The same generic request, denim jeans, can therefore produce a different ordering for different customers.
Two signals now participate in ranking:
| Signal | Evidence |
|---|---|
| Search score | Similarity between query and product text vectors |
| Purchase score | Candidate popularity through peer-group purchase paths |
The demonstration combines the signals multiplicatively. Its reordered denim results include Rachel HW denim and Jade HW denim skinny; items overlap with the original semantic results, but their positions change.
OPTIONAL MATCH preserves a semantic candidate even when no corresponding peer-group purchase path exists. This matters when a customer has no purchase history, or when their peers have never bought the kind of item being searched for. Preserving the row and preserving a useful ranking are separate concerns: multiplying a raw zero purchase score by a search score produces zero, so vector-only behavior requires the scoring expression or ordering to retain the semantic signal as well. Zach describes that vector-only behavior as the intended fallback. He also offers the reverse ordering—graph traversal followed by vector comparison—as another retrieval strategy.
An external batch recommender could supply candidates instead. The reason to traverse the transactional graph at request time is freshness and flexibility: a new purchase can affect the next query, and changing the time window or traversal depth changes the evidence used without requiring a fixed recommendation list to be rebuilt first.
The audience challenges the multiplication rule because the two scores have different distributions and scales. Zach treats it as a starting heuristic, suggests rank-based ordering as another approach, and recommends evaluating alternatives against labeled recommendation outcomes. A useful production comparison asks which ranking method historically led to successful recommendations, rather than assuming that a plausible formula is calibrated.
The filtering alternatives depend on how much useful behavior the graph contains:
- Graph first: Restrict candidates to the customer’s neighborhood, then apply a semantic threshold. This can be too restrictive if that neighborhood contains no suitable denim products.
- Vector first: Retrieve semantically relevant products, then use graph evidence to filter or rerank them. Retaining vector results when the graph has no match helps with sparse histories.
- Dense purchase history: A well-connected neighborhood makes graph-first restriction more plausible; sparse history favors optional matching and less aggressive filtering.
An attendee also proposes reciprocal rank fusion. Zach says that specific query-time capability is not available in the demonstrated setup; the GDS normalization functions he mentions do not provide it there. The next stage moves from explicit purchase paths to inferred relationships.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn co-purchase structure into reusable recommendation links
This workshop starts with structured data; extracting a knowledge graph from unstructured documents is a separate workflow. Here, Graph Data Science supplies a workspace for projecting selected graph data into an efficient sparse-matrix representation. Its broader algorithm families include pathfinding, centrality, community detection, topological link prediction and directed acyclic graph algorithms. The session narrows that toolkit to node embeddings and similarity.
A visualization of Zachary’s Karate Club introduces the intuition: nearby groups in the graph appear near one another in a two-dimensional embedding. The workshop calls this locality homophily. For retail data, article embeddings encode co-purchase structure, and nearest-neighbor computation uses those representations to infer useful article-to-article connections before a request arrives. Entity resolution and person–location inference are other possible applications.
Embedding nodes does not discard edges. Relationships determine the neighborhoods that place nodes in vector space. The output is a vector for each node, but the evidence used to construct it is relational. Embeddings of head–relationship–tail triples or whole subgraphs are other representations; they are not the one built here.
The notebook constructs the recommendation layer in four steps:
- Project co-purchases. Follow shared purchases through customers, then create an in-memory graph of articles connected by weighted co-purchase relationships.
- Compute article embeddings. Run FastRP, fast random projection. Sparse random-matrix operations encode neighborhood structure without requiring a neural network. The workshop uses 128-dimensional graph embeddings, with iteration weights controlling the contributions from successive neighborhoods.
- Find similar articles. Run
gds.knn.writeto write scored customer-also-likes relationships between articles with similar embeddings. Sampling reduces the nearest-neighbor computation required. - Persist and inspect. Write embeddings back as needed and inspect the algorithm’s convergence and relationship-write statistics.
These new links are derived recommendations, not records of purchases that occurred. The spoken cutoff is tentative; the useful distinction is between raw cosine and the GDS kNN cosine score, which the documentation normalizes to the interval from zero to one.
The notebook also contains a t-SNE visualization that reduces the graph embeddings to two dimensions. Its computation cells are commented out because they take time, while the retained visualization shows clusters of commonly purchased articles. These are co-purchasing clusters, not necessarily shopping baskets: explicitly modeling orders would be needed to distinguish items purchased together in a particular basket.
Retrieval can now follow a short path: customer → purchased article → inferred similar article → product. This recommendation query needs a customer’s history but no search prompt. The expensive similarity work has already produced traversable relationships. A vector index over graph embeddings would be another design, but this demonstration precomputes kNN instead. Supervised or external models could generate the inferred links, and date restrictions could narrow the purchase history used to retrieve them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Refresh embeddings and choose their size
The demonstrated graph-embedding workflow requires recalculation as the graph changes. Zach describes more immediate updates as future work, while suggesting that averaging a new node’s neighbors could provide an interim representation before recomputation. He reports seeing FastRP process graphs with billions of nodes and relationships in about half an hour, without specifying hardware, dimensions or the precise workload. That anecdote cannot determine a refresh schedule for this application; the relevant operational question is how quickly changing purchase behavior must reach the persisted recommendation links.
The demo graph contains 1,000 customers. Its small embedding size is chosen for the sandbox, and Zach suggests trying 256 dimensions or larger sizes as the dataset grows. Powers of two are conventional choices, not a requirement; FastRP accepts a positive integer dimension, and memory and runtime increase with it. His statement that the product has no inherent upper dimension limit should therefore be read alongside those practical costs. He also observes that graph embeddings often need fewer dimensions than general-purpose language embeddings.
Asked about deployed graph-embedding use cases, Zach names entity resolution, fraud-related classification, recommendation and customer segmentation. Those uses share a need to represent behavioral or structural relationships as features, even when the downstream task is not product search.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use graph queries for questions similarity cannot answer
How many types of pants are in the catalog? That is an aggregation question, not a request for the nearest descriptions. Text-to-Cypher can generate a query to answer it; predefined Cypher templates or snippets can serve the same purpose. A semantic layer can route the question to the appropriate tool. The customer-count result visible in Browser illustrates the distinction: a graph query computes an aggregate directly rather than inferring it from retrieved examples.
Function calling makes those operations available to a model that supports tools. Descriptions and input definitions guide its selection. Generic operations might retrieve a node’s neighborhood, find a shortest path, or find similar nodes using graph embeddings. Domain experts can add legal or fintech queries with more specific semantics. The application then offers a bounded menu of graph operations rather than asking every question to pass through one vector-search routine.
A final question revisits why graph embeddings are useful when they are computed from the application’s own graph rather than pretrained across internet-scale text or images. Their purpose here is to encode multi-hop topology into an intermediate representation. Similarity operations can reuse that representation instead of repeating an equivalent traversal for every request. In an identity graph, for example, relationships among people, identifiers and transactions might supply features for inferring a shared household. The graph’s topology becomes the input to similarity and link inference, and the resulting vectors can support several downstream operations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Feed two retrieved product lists into one email prompt
The final stage gives the language model a constrained writing task. The prompt assigns an assistant identity and company, asks for an engaging email suited to the season and the customer’s interests, and instructs it to select only from supplied products. Those products arrive in two lists: personalized search results and additional graph-derived recommendations. The model has freedom to select and describe items within that supplied context.
The application accepts four inputs: search prompt, customer ID, customer name and time of year. Query text and customer ID feed personalized search; the customer ID separately feeds recommendation lookup. Both retrieved lists, together with the display name, season and interests, become prompt values. The distinction between the branches is useful:
| Retriever | Inputs | Evidence |
|---|---|---|
| Personalized search | Query and customer ID | Text similarity plus peer-group purchases |
| Recommendations | Customer ID | Purchase history plus inferred article links |
One branch answers the current request; the other supplies additional products based on behavior.
LangChain instantiates GPT-4o, wraps personalized retrieval so it can receive a customer ID, and exposes a separate recommendation function. Each branch formats its results for the prompt. The following Python composition captures that boundary: personalized_search accepts a query and customer ID, while recommendations accepts a customer ID; both return formatted product context.
python
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda
from langchain_openai import ChatOpenAI
def build_email_chain(personalized_search, recommendations):
def retrieve(request):
return {
"customer_name": request["customer_name"],
"time_of_year": request["time_of_year"],
"interests": request["search_prompt"],
"search_products": personalized_search(
request["search_prompt"], request["customer_id"]
),
"recommendation_products": recommendations(
request["customer_id"]
),
}
prompt = ChatPromptTemplate.from_template(
"Write an engaging product email to {customer_name}.\n"
"Time of year: {time_of_year}\n"
"Customer interests: {interests}\n"
"Use only products supplied below. Select relevant items "
"for the season and preserve their supplied links.\n\n"
"Search products:\n{search_products}\n\n"
"Additional recommendations:\n{recommendation_products}"
)
return RunnableLambda(retrieve) | prompt | ChatOpenAI(model="gpt-4o")
The LangChain Expression Language pipe carries the prepared values into prompt construction and then into the model. The customer ID guides retrieval; the five prompt values are the two product lists, customer name, time of year and interests.
The notebook invokes the chain for denim jeans, a customer ID, Alex’s name and a summer time of year. The name is supplied separately because the retail data is anonymized. The output contains a subject line, greeting, five jean products and complementary summer items. Search supplies the jeans; graph-derived recommendations supply the additional choices. The same prompt structure could be narrowed to search-only or recommendation-only content.
Inspecting the full formatted prompt reveals what the model actually received: product descriptions and metadata from both retrieval branches. This is the place to separate a retrieval problem from a writing problem—whether the relevant item was supplied at all, and then whether the model chose it. Zach states that the prompt receives 10 search products and 100 recommendation products. Increasing either list eventually encounters the model’s context-window limit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Change the customer, then change the season
A final notebook cell launches the Gradio interface, with preset examples and a link to the running demo. It exposes the same inputs already used by the chain. Zach estimates a normal response time of 10–15 seconds and expects roughly 20 seconds on the room’s slow connection. The displayed form shows Alex Smith, June 2024 and denim jeans beside a completed summer recommendation email.
Keeping denim jeans fixed while changing the customer ID changes the purchase neighborhood used by retrieval. The demonstration returns different recommendations because the customers occupy different parts of the purchase graph. Similar histories can legitimately produce similar results; changing an identifier is not by itself a reason to force a different answer.
Changing the time of year to February exercises a different part of the system. The prompt now asks the language model for winter framing and seasonally appropriate selections, such as sweaters, from the supplied products. That change does not alter the graph. The completed demo makes the responsibilities visible: purchase relationships determine personalized evidence, inferred article links expand the recommendation set, and the language model turns those retrieved choices into a seasonal email.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Guide to the H&M retail workshop, its preparation and demo notebooks, and version history.
Further reading
Explains how FastRP encodes neighborhoods and how dimensions, iteration weights and relationship weights affect its results.
Documents similarity metrics, sampling and writing nearest-neighbor relationships back to Neo4j.
Updates since the talk
Maintained notebook for the customer-and-product GraphRAG workshop; setup has changed since the recording.
Reference for vector-index dimensions, similarity scores and quantization, including changes after the workshop.
Full transcript
Loading transcript…