AI Engineer World's Fair 2025

Stop Using RAG as Memory

Loading slides…

Read the talk

Stop Using RAG as Memory

A media assistant that recalls a dog named Melody when asked for favorite tunes exposes the gap between semantic similarity and domain-aware agent memory.

From a talk by Daniel Chalef

Before you start: Familiarity with embeddings, agent tools and basic TypeScript will help you follow the retrieval and schema examples.

What should an assistant remember?

What should a media assistant remember about its user? The answer depends on what the assistant is supposed to do. Memory needs a business domain: a useful representation for a media player will differ from one for a financial coach.

Graphiti, Zep’s open-source temporal graph framework, supports custom entities and edges that represent a domain’s business objects. Daniel Chalef demonstrates how Zep exposes that approach to developers working in Python, TypeScript or Go. The developer supplies the kinds of things the application needs to remember, rather than treating every extracted fact as equally useful memory.

Chalef starts with a relevance problem he has observed in ChatGPT: memory can surface arbitrary personal facts, contributing to inaccurate responses or hallucinations. His media-assistant example makes the problem concrete. Jazz, NPR podcasts and The Daily are useful listening preferences. But conversation—and potentially incidental speech picked up by a voice agent—also supplies facts such as waking up at 7 AM and having a dog named Melody. Those facts describe the same person without necessarily helping the media player perform its job.

Slide listing jazz, NPR, The Daily and a paused Taylor Swift song alongside purple entries about waking time, the gym, mornings, a dog and recipes.Open full source frame
A media assistant mixes listening preferences with unrelated personal facts.
0:000:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Why Melody appears among favorite tunes

The failure follows from a straightforward memory pipeline: extract facts, store them in a vector database or Redis, and retrieve facts that are semantically similar to the next request. That pipeline gives retrieval a similarity criterion, but no explicit account of which facts belong to the application’s domain.

In Chalef’s illustration, asking for favorite tunes retrieves useful listening preferences alongside the dog named Melody. The word has a musical association; the fact concerns a pet. A plausible semantic match has crossed the application’s relevance boundary.

Semantic similarity is not business relevance. Chalef describes vectors as projections into an embedding space: proximity does not itself establish an explicit causal or relational connection between the represented facts. Improving similarity ranking alone does not supply the missing domain model. The proposed remedy is domain-aware memory, with structure that lets the application distinguish the kinds of information it needs. Semantic search can still serve retrieval within those boundaries.

2:282:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:28 · section reference included

Give the finance coach explicit business objects

The recorded demonstration switches to a finance coach. The user wants to buy a house, so the assistant asks about annual earnings and student-loan debt. Beside the conversation, Zep’s memory contains explicit business objects: financial goals, debts and income sources. These are developer-defined categories, giving the application a vocabulary for the information its advice depends on.

Developers can express these schemas using Pydantic, Zod or Go structs. Chalef opens a TypeScript financial-goal schema built with Zep’s SDK, then shows how entity descriptions and field descriptions express what should be extracted and what values fields should contain. These descriptions guide extraction; describing a business rule is distinct from enforcing it through application validation.

For a house-purchase goal, a small Zod schema illustrates that separation. Descriptions explain the fields’ meaning, while .positive() supplies a concrete validation constraint when the application parses a value:

typescript

import { z } from "zod";

const FinancialGoal = z.object({
  description: z.string().describe(
    "The financial objective, such as buying a house"
  ),
  targetAmount: z.number().positive().optional().describe(
    "The amount needed for the goal, when stated by the user"
  ),
}).describe("A financial objective the user wants to achieve");

type FinancialGoal = z.infer<typeof FinancialGoal>;

const goal: FinancialGoal = FinancialGoal.parse({
  description: "Buy a house",
});

The optional amount allows the goal to exist before the user supplies a target. The demonstrated schema view extends this descriptive approach to assets and income sources, including income type, monthly amount and reliability.

Code editor showing asset fields and an income-source schema with descriptions for income type, monthly amount and reliability.Open full source frame
Entity definitions describe asset and income-source fields.
4:134:23
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:13 · section reference included

Retrieve by type, then register the ontology

With typed objects available, the agent’s financial-snapshot tool runs multiple Zep searches concurrently and filters each search by a specific node type. The displayed code searches for financial goals, expense categories and debt accounts. Each search has a defined category of eligible results instead of relying on one broad similarity query to separate every kind of fact.

Concurrency and filtering solve different problems. Concurrency lets the independent searches proceed together; node-type filters narrow what each search can return. An agent can then assemble the separate results into a financial snapshot while preserving their meanings as goals, expenses and debts.

Code showing concurrent graph searches with node-label filters for financial goals, expense categories and debt accounts.Open full source frame
Financial snapshot code searches for specific entity types.

The next code view supplies the setup that makes those types available: at application startup, the developer registers the business objects with Zep so it knows which ontology to build in the graph. Schema definitions establish the vocabulary; registration makes that vocabulary available to the memory system; typed retrieval uses it to select context for the agent.

5:275:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:27 · section reference included

Follow a new expense into memory

Chalef adds a monthly amount of $5,000, describing it as rent and then qualifying that with “I think it’s rent.” In the prerecorded demonstration, Chalef reports that Zep parses the new message and captures the $5,000 amount within a few seconds. The visible result is a memory panel whose Expenses section shows a monthly amount of $5,000 beside the conversation. That timing describes this demonstration, with no measured workload or independent latency test supplied.

Finance chat displaying “$5000 a month” beside a memory panel with Income Sources and Expenses sections; the expense entry shows Monthly: $5,000.Open full source frame
The memory panel now includes the new expense.

He then opens Zep’s front end to inspect the user’s knowledge graph. It contains a debt-account entity with developer-defined fields. This graph view demonstrates the structured representation; it does not establish that the debt account is the newly entered rent expense.

The practical endpoint is control over retrieval: once memory has explicit types and fields, the application can filter tightly for the objects a task needs. Chalef closes by pointing viewers to a paper explaining the system. The demonstration ends with the full connection in place—from a conversational message, to structured memory, to selective retrieval for the next agent action.

5:586:03
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:58 · section reference included

Resources

From the talk

  • Open-source framework for temporal knowledge graphs, with installation instructions and examples for ingestion and hybrid retrieval.

Updates since the talk

  • Current instructions for defining and registering entity and edge types, filtering retrieval, and configuring strict ontology extraction.

Transcript download
Full transcript

Loading transcript…