AI Engineer World's Fair 2024

Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications

Loading slides…

Read the talk

Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications

Build a retail graph, search product descriptions, personalize results through purchase paths, and use graph embeddings to supply recommendations for a seasonal email.

From a talk by Andreas Kollegger, Zach Blumenfeld, Michael Hunger and Tomasz

Before you start: You should be comfortable reading Python notebooks and basic database queries; the article introduces the graph and vector-search concepts used in the workshop.

Prepare the notebook and graph database

What needs to be running before a Python notebook can retrieve real retail data from a graph? The workshop starts with two pieces: a blank Neo4j Graph Data Science sandbox and an editable notebook. Zach leads the implementation, with Andreas, Michael Hunger and Tomasz helping participants through the alternating slides and notebook exercises.

  1. Create a sandbox by choosing For Data Science, then the blank Graph Data Science option. Keep its generated connection credentials.
  2. Open the GenAI Workshop notebook from the workshop repository. In Colab, use File → Save a Copy in Drive so you can edit it. VS Code or a local notebook environment is also suitable.
  3. Run the dependency installation cell early; loading the packages takes time.

These are the recorded setup steps. The maintained workshop materials have since moved to AuraDS and reorganized the notebooks, so their current setup differs from the recording.

Before We Start slide with sandbox and GitHub links, a QR code, and instructions to open genai-workshop.ipynb.Open full source frame
Workshop prerequisites: a blank Neo4j GDS Sandbox and the Colab notebook.

If you accidentally create the regular blank sandbox, the workshop’s correction is to terminate that instance and create the GDS variant. The regular instance should still work, but the GDS sandbox has more memory for the heavier operations that come later.

0:260:40
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:26 · section reference included

Find an item, personalize it, then suggest what goes with it

Retrieval-augmented generation inserts a data lookup between a user’s request and a language model’s response. That lookup supplies domain context and traceable source material. Here, GraphRAG combines vector search, graph traversal and graph data science: descriptions identify relevant products, purchase relationships personalize them, and graph embeddings provide additional recommendations.

Consider a customer looking for a halter neck top. First, the application needs relevant items from the store’s catalog. Then it needs to choose among those items for this particular customer. Finally, it may suggest pants, a hat or accessories to pair with the top. The workshop uses H&M competition data to build that pipeline and lets the language model turn its results into an email. The same separation between retrieval and composition can serve support agents, internal search and other customer experiences.

The repository also contains workshop slides, although Zach notes that the PDF available during the session differs from his live deck. The finished interface is a small Gradio app accepting customer information, interests and a time of year. OpenAI supplies the language model, LangChain orchestrates the vector stores and generation chain, and Colab hosts the notebook; other model providers could fill the generation role. The implementation proceeds from graph construction to vector search, purchase-based personalization, graph embeddings and nearest neighbors, and finally email generation.

5:596:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:59 · section reference included

Build the customer–article–product graph

The data model separates a product from its sellable variants. A customer purchases an Article, and that article is a variant of a Product. A jeans style is the product; its colors, patterns or sizes distinguish variants. Articles also connect to departments, though the workshop makes little use of department nodes.

EntityRelevant properties
CustomerUnique customer ID
ArticleColor and design pattern
ProductName, description, type and garment group
DepartmentDepartment association

This separation lets purchase behavior remain specific to articles while semantic search operates on product descriptions.

After installing packages and importing the notebook’s libraries, copy your sandbox’s Bolt connection URL into the Neo4j URI setting, supply its password, and retain neo4j as the username. The generation model is GPT-4o. Use your own OpenAI credentials; the key offered in the session was temporary. The notebook also offers a separate environment-file route for configuration. The Graph Data Science client connects with the URI and credentials, and the following debug cell checks that connection.

The preparation notebook supplies five CSVs: department, product, article, customer and transaction. The first four describe entities; transactions supply customer-to-article purchase relationships. The loader creates uniqueness constraints, loads nodes, then loads relationships. Its generated Cypher uses UNWIND, MERGE and SET: expand input rows, identify or create the corresponding graph objects, and populate their properties. Zach points to GraphAcademy for a deeper treatment of graph modeling and ingestion.

Neo4j Browser provides an immediate inspection surface. You can sign in with the database username and password instead of single sign-on. The database panel lists node labels, relationship types and properties; selecting the Article label runs a MATCH query displaying a small sample of nodes.

Two finishing steps prepare products for retrieval and output. Because the dataset lacks product links, the loader creates placeholder URLs; links generated from them are not real shopping destinations. It also combines product name, type, garment group and description into one text property. That combined field becomes the input to text embedding.

11:5612:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:56 · section reference included

Wrap retrieval without coupling it to one storage layout

An attendee encounters an unknown-property warning, but no diagnosis is established in the discussion. A separate question clarifies the index structure: the HNSW graph is internal to each vector index, distinct from the customer–article–product graph. The recorded implementation uses Lucene underneath that index.

Existing embeddings can be imported with dimensions matching the index. Zach points to a commented LangChain example for external embedding generation, while noting that he has not run that block recently. Keeping vector retrieval inside Neo4j makes it convenient to continue directly into graph traversal, but it is not required: an orchestration layer can query an external vector store and then use Neo4j for connected-data retrieval.

The graph becomes useful when the application needs structured personalization or graph-derived embeddings beyond plain text search. It can also persist precomputed nearest-neighbor relationships for later lookup. Zach cites index-free adjacency, expressive multi-hop patterns and a flexible schema as reasons to use a graph database. His answer that scalar quantization is not yet available belongs to the recording: the vector-index manual documents quantization configuration introduced in Neo4j 5.23.

LangChain’s Neo4jVector now wraps the existing index using its name, the database credentials and an OpenAI embedding object. The workshop uses 1,536-dimensional text embeddings. This wrapper returns documents containing product descriptions, URLs and metadata, which can also be displayed as a dataframe. Searching for denim jeans reproduces the earlier retrieval; changing the prompt to oversized sweaters produces matching sweater descriptions. The wrapper prepares the same retrieval operation for the eventual generation chain.

Colab cell containing an oversized sweaters similarity search above a dataframe of sweater descriptions.Open full source frame
Searching for oversized sweaters returns matching product descriptions.

The scaling discussion does not establish a fixed capacity limit or automatic-sharding guarantee. Asked about billions of documents, Zach describes indexes instantiated over selected properties, metadata-based federation being explored, and an external vector store as another architecture for very large collections. The application can replace the vector-retrieval call while retaining its Neo4j traversal stage.

35:4735:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

35:47 · section reference included

Derive a peer group from shared purchases

Vector search finds relevant descriptions. Graph pattern matching then follows the entities associated with those descriptions. In this dataset, customers who purchase similar articles provide evidence about what one another might want next—the familiar collaborative filtering pattern expressed as a traversal.

Browser makes the relationships concrete. Its schema view shows customers, articles, products and departments; an additional inferred relationship visible in the presenter’s database has not yet been created in the participants’ databases. A sample query follows products to their article variants and then to purchasing customers. In the displayed graph, customers are orange, articles blue and products red. Purchase relationships carry dates and prices, while product nodes expose the assembled text and stored OpenAI embeddings.

The next query fixes a customer ID as a reusable parameter and retrieves that customer’s purchase history, including product names, types and descriptions. From there, the recommendation traversal proceeds in order:

  1. Find the customer’s latest purchase date.
  2. Select the articles purchased on that date.
  3. Find other customers who purchased those articles.
  4. Follow those customers’ purchases to additional articles.
  5. Return candidates the original customer has not already bought.

Zach illustrates the idea with ABK: if they buy similar things, ABK’s other purchases may reveal something Zach would like.

The resulting common purchase score counts matching co-purchase paths. It measures how often a candidate appears through the selected peer group; it is not itself a probability of purchase. Sweaters and T-shirts appear among the ranked candidates. The peer group is derived from shared purchases, rather than assigned as a demographic category: start at a customer, cross to a purchased article, cross back to its other buyers, then follow their other purchases. More matching paths give the candidate more support under this rule.

Notebook query showing latest-purchase selection, customer purchase paths, product fields, and ordering by commonPurchaseScore.Open full source frame
A Cypher query finds related products through shared purchase paths.

The traversal can extend farther, and its time scope is adjustable. The demonstration selects the maximum purchase date. A condition on t_dat, the purchase-date field, could instead select the last week or the last two weeks. That choice determines which behavior defines the customer’s current neighborhood.

45:1845:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

45:18 · section reference included

Rerank semantic matches without losing cold-start results

A custom retrieval_query on Neo4jVector appends the purchase-pattern logic after vector search. The application passes a customer ID along with the query, finds that customer’s peer group, and scores how strongly each retrieved product appears in it. The same generic request, denim jeans, can therefore produce a different ordering for different customers.

Two signals now participate in ranking:

SignalEvidence
Search scoreSimilarity between query and product text vectors
Purchase scoreCandidate popularity through peer-group purchase paths

The demonstration combines the signals multiplicatively. Its reordered denim results include Rachel HW denim and Jade HW denim skinny; items overlap with the original semantic results, but their positions change.

OPTIONAL MATCH preserves a semantic candidate even when no corresponding peer-group purchase path exists. This matters when a customer has no purchase history, or when their peers have never bought the kind of item being searched for. Preserving the row and preserving a useful ranking are separate concerns: multiplying a raw zero purchase score by a search score produces zero, so vector-only behavior requires the scoring expression or ordering to retain the semantic signal as well. Zach describes that vector-only behavior as the intended fallback. He also offers the reverse ordering—graph traversal followed by vector comparison—as another retrieval strategy.

An external batch recommender could supply candidates instead. The reason to traverse the transactional graph at request time is freshness and flexibility: a new purchase can affect the next query, and changing the time window or traversal depth changes the evidence used without requiring a fixed recommendation list to be rebuilt first.

The audience challenges the multiplication rule because the two scores have different distributions and scales. Zach treats it as a starting heuristic, suggests rank-based ordering as another approach, and recommends evaluating alternatives against labeled recommendation outcomes. A useful production comparison asks which ranking method historically led to successful recommendations, rather than assuming that a plausible formula is calibrated.

Colab code defining personalized retrieval with a purchase-path match, a multiplied score, metadata, and descending ordering.Open full source frame
The retrieval query combines purchase and search scores.

The filtering alternatives depend on how much useful behavior the graph contains:

  • Graph first: Restrict candidates to the customer’s neighborhood, then apply a semantic threshold. This can be too restrictive if that neighborhood contains no suitable denim products.
  • Vector first: Retrieve semantically relevant products, then use graph evidence to filter or rerank them. Retaining vector results when the graph has no match helps with sparse histories.
  • Dense purchase history: A well-connected neighborhood makes graph-first restriction more plausible; sparse history favors optional matching and less aggressive filtering.

An attendee also proposes reciprocal rank fusion. Zach says that specific query-time capability is not available in the demonstrated setup; the GDS normalization functions he mentions do not provide it there. The next stage moves from explicit purchase paths to inferred relationships.

57:3957:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

57:39 · section reference included

Turn co-purchase structure into reusable recommendation links

This workshop starts with structured data; extracting a knowledge graph from unstructured documents is a separate workflow. Here, Graph Data Science supplies a workspace for projecting selected graph data into an efficient sparse-matrix representation. Its broader algorithm families include pathfinding, centrality, community detection, topological link prediction and directed acyclic graph algorithms. The session narrows that toolkit to node embeddings and similarity.

A visualization of Zachary’s Karate Club introduces the intuition: nearby groups in the graph appear near one another in a two-dimensional embedding. The workshop calls this locality homophily. For retail data, article embeddings encode co-purchase structure, and nearest-neighbor computation uses those representations to infer useful article-to-article connections before a request arrives. Entity resolution and person–location inference are other possible applications.

Embedding nodes does not discard edges. Relationships determine the neighborhoods that place nodes in vector space. The output is a vector for each node, but the evidence used to construct it is relational. Embeddings of head–relationship–tail triples or whole subgraphs are other representations; they are not the one built here.

The notebook constructs the recommendation layer in four steps:

  1. Project co-purchases. Follow shared purchases through customers, then create an in-memory graph of articles connected by weighted co-purchase relationships.
  2. Compute article embeddings. Run FastRP, fast random projection. Sparse random-matrix operations encode neighborhood structure without requiring a neural network. The workshop uses 128-dimensional graph embeddings, with iteration weights controlling the contributions from successive neighborhoods.
  3. Find similar articles. Run gds.knn.write to write scored customer-also-likes relationships between articles with similar embeddings. Sampling reduces the nearest-neighbor computation required.
  4. Persist and inspect. Write embeddings back as needed and inspect the algorithm’s convergence and relationship-write statistics.

These new links are derived recommendations, not records of purchases that occurred. The spoken cutoff is tentative; the useful distinction is between raw cosine and the GDS kNN cosine score, which the documentation normalizes to the interval from zero to one.

The notebook also contains a t-SNE visualization that reduces the graph embeddings to two dimensions. Its computation cells are commented out because they take time, while the retained visualization shows clusters of commonly purchased articles. These are co-purchasing clusters, not necessarily shopping baskets: explicitly modeling orders would be needed to distinguish items purchased together in a particular basket.

Retrieval can now follow a short path: customer → purchased article → inferred similar article → product. This recommendation query needs a customer’s history but no search prompt. The expensive similarity work has already produced traversable relationships. A vector index over graph embeddings would be another design, but this demonstration precomputes kNN instead. Supervised or external models could generate the inferred links, and date restrictions could narrow the purchase history used to retrieve them.

1:09:331:09:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:09:33 · section reference included

Refresh embeddings and choose their size

The demonstrated graph-embedding workflow requires recalculation as the graph changes. Zach describes more immediate updates as future work, while suggesting that averaging a new node’s neighbors could provide an interim representation before recomputation. He reports seeing FastRP process graphs with billions of nodes and relationships in about half an hour, without specifying hardware, dimensions or the precise workload. That anecdote cannot determine a refresh schedule for this application; the relevant operational question is how quickly changing purchase behavior must reach the persisted recommendation links.

The demo graph contains 1,000 customers. Its small embedding size is chosen for the sandbox, and Zach suggests trying 256 dimensions or larger sizes as the dataset grows. Powers of two are conventional choices, not a requirement; FastRP accepts a positive integer dimension, and memory and runtime increase with it. His statement that the product has no inherent upper dimension limit should therefore be read alongside those practical costs. He also observes that graph embeddings often need fewer dimensions than general-purpose language embeddings.

Asked about deployed graph-embedding use cases, Zach names entity resolution, fraud-related classification, recommendation and customer segmentation. Those uses share a need to represent behavioral or structural relationships as features, even when the downstream task is not product search.

1:20:071:20:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:20:07 · section reference included

Use graph queries for questions similarity cannot answer

How many types of pants are in the catalog? That is an aggregation question, not a request for the nearest descriptions. Text-to-Cypher can generate a query to answer it; predefined Cypher templates or snippets can serve the same purpose. A semantic layer can route the question to the appropriate tool. The customer-count result visible in Browser illustrates the distinction: a graph query computes an aggregate directly rather than inferring it from retrieved examples.

Neo4j Browser displaying a Customer count query and a single result of 1000.Open full source frame
A customer-count query returns 1000.

Function calling makes those operations available to a model that supports tools. Descriptions and input definitions guide its selection. Generic operations might retrieve a node’s neighborhood, find a shortest path, or find similar nodes using graph embeddings. Domain experts can add legal or fintech queries with more specific semantics. The application then offers a bounded menu of graph operations rather than asking every question to pass through one vector-search routine.

A final question revisits why graph embeddings are useful when they are computed from the application’s own graph rather than pretrained across internet-scale text or images. Their purpose here is to encode multi-hop topology into an intermediate representation. Similarity operations can reuse that representation instead of repeating an equivalent traversal for every request. In an identity graph, for example, relationships among people, identifiers and transactions might supply features for inferring a shared household. The graph’s topology becomes the input to similarity and link inference, and the resulting vectors can support several downstream operations.

1:23:441:23:59
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:23:44 · section reference included

Feed two retrieved product lists into one email prompt

The final stage gives the language model a constrained writing task. The prompt assigns an assistant identity and company, asks for an engaging email suited to the season and the customer’s interests, and instructs it to select only from supplied products. Those products arrive in two lists: personalized search results and additional graph-derived recommendations. The model has freedom to select and describe items within that supplied context.

The application accepts four inputs: search prompt, customer ID, customer name and time of year. Query text and customer ID feed personalized search; the customer ID separately feeds recommendation lookup. Both retrieved lists, together with the display name, season and interests, become prompt values. The distinction between the branches is useful:

RetrieverInputsEvidence
Personalized searchQuery and customer IDText similarity plus peer-group purchases
RecommendationsCustomer IDPurchase history plus inferred article links

One branch answers the current request; the other supplies additional products based on behavior.

LangChain instantiates GPT-4o, wraps personalized retrieval so it can receive a customer ID, and exposes a separate recommendation function. Each branch formats its results for the prompt. The following Python composition captures that boundary: personalized_search accepts a query and customer ID, while recommendations accepts a customer ID; both return formatted product context.

python

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda
from langchain_openai import ChatOpenAI


def build_email_chain(personalized_search, recommendations):
    def retrieve(request):
        return {
            "customer_name": request["customer_name"],
            "time_of_year": request["time_of_year"],
            "interests": request["search_prompt"],
            "search_products": personalized_search(
                request["search_prompt"], request["customer_id"]
            ),
            "recommendation_products": recommendations(
                request["customer_id"]
            ),
        }

    prompt = ChatPromptTemplate.from_template(
        "Write an engaging product email to {customer_name}.\n"
        "Time of year: {time_of_year}\n"
        "Customer interests: {interests}\n"
        "Use only products supplied below. Select relevant items "
        "for the season and preserve their supplied links.\n\n"
        "Search products:\n{search_products}\n\n"
        "Additional recommendations:\n{recommendation_products}"
    )
    return RunnableLambda(retrieve) | prompt | ChatOpenAI(model="gpt-4o")

The LangChain Expression Language pipe carries the prepared values into prompt construction and then into the model. The customer ID guides retrieval; the five prompt values are the two product lists, customer name, time of year and interests.

The notebook invokes the chain for denim jeans, a customer ID, Alex’s name and a summer time of year. The name is supplied separately because the retail data is anonymized. The output contains a subject line, greeting, five jean products and complementary summer items. Search supplies the jeans; graph-derived recommendations supply the additional choices. The same prompt structure could be narrowed to search-only or recommendation-only content.

Inspecting the full formatted prompt reveals what the model actually received: product descriptions and metadata from both retrieval branches. This is the place to separate a retrieval problem from a writing problem—whether the relevant item was supplied at all, and then whether the model chose it. Zach states that the prompt receives 10 search products and 100 recommendation products. Increasing either list eventually encounters the model’s context-window limit.

1:29:301:29:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:29:30 · section reference included

Change the customer, then change the season

A final notebook cell launches the Gradio interface, with preset examples and a link to the running demo. It exposes the same inputs already used by the chain. Zach estimates a normal response time of 10–15 seconds and expects roughly 20 seconds on the room’s slow connection. The displayed form shows Alex Smith, June 2024 and denim jeans beside a completed summer recommendation email.

Gradio form showing Alex Smith, June 2024, and denim jeans beside a generated email with linked product recommendations.Open full source frame
The demo returns a summer denim recommendation email for Alex Smith.

Keeping denim jeans fixed while changing the customer ID changes the purchase neighborhood used by retrieval. The demonstration returns different recommendations because the customers occupy different parts of the purchase graph. Similar histories can legitimately produce similar results; changing an identifier is not by itself a reason to force a different answer.

Changing the time of year to February exercises a different part of the system. The prompt now asks the language model for winter framing and seasonally appropriate selections, such as sweaters, from the supplied products. That change does not alter the graph. The completed demo makes the responsibilities visible: purchase relationships determine personalized evidence, inferred article links expand the recommendation set, and the language model turns those retrieved choices into a seasonal email.

1:37:041:37:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:37:04 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…