AI Engineer World's Fair 2024
Hasura Launch: Realtime Data Connectivity for AI
Loading slides…
Read the talk
Hasura Launch: Realtime Data Connectivity for AI
A customer email becomes a data-access problem: find the right customer, retrieve their recent activity, enforce permissions, and give the model a way to assemble the answer.
From a talk by Tanmai Gopal
Before you start: Basic familiarity with LLM tool use, APIs, and database queries will help; the SQL and Python examples require no Hasura-specific knowledge.
Why can an LLM build a game but not answer a calendar question?
Connecting an AI to live data sounds simpler than asking it to build an application. Tanmai Gopal opens with a joke about giving the coming AI overlords enough data to do their jobs, then promises an explanation simple enough to stake his switch from GPT-4 to Sonnet on it. The practical frustration is familiar: Gopal says an LLM can make a Flappy Bird game featuring his face in 30 seconds, yet cannot intelligently answer questions about his own data. That timing is his personal anecdote, with no model or measurement conditions specified.
The questions are ordinary, but each requires more than generating plausible text:
- Calendar: How many one-on-ones happened last week? What frequency makes sense given the team’s roles, and how should future meetings be staggered?
- Salesforce: Why is the Acme deal stuck in stage three? Answering requires understanding what should happen between stages two and three and finding what is missing for this deal.
- Support and product data: Is a ticket from an enterprise customer? Which project does it concern, what is that project’s status, and where is it in the product funnel?
These requests move from retrieving records to interpreting those records through the business’s relationships and rules.
Even a shopping assistant embedded beside a product can miss the relevant connection. Gopal describes asking Amazon’s Rufus whether a product could reach his delivery destination with one-day delivery. The slide shows a showerhead listing and the delivery question; he reports that Rufus could not answer it in that interaction. The necessary answer depends on the product, the destination, and delivery availability together.
Connecting more systems also raises a trust problem. Gopal does not want to expose his calendar through a custom GPT without knowing what happens to the data. Useful connectivity must include controlled access, rather than merely making another source reachable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make live data and business logic available as a tool
The proposed interface is straightforward: expose live data and business logic as a tool the LLM can use. The model needs a way to obtain the information required for a request, including the business logic that gives that information meaning. The interesting engineering lies in making that interface work across sources while preserving access rules.
The demonstration uses a fictional Blockbuster business, introduced with a joke about services businesses surviving while movie streaming goes nowhere in the AI world. Its database and transactions support a request to draft an email thanking the top customer for their patronage and mentioning some movies they watched recently.
That short request implies a retrieval procedure:
- Identify the top customer from the available business data.
- Retrieve that customer’s recent movie history.
- Use those facts to draft a personalized thank-you email.
Gopal reports that the request works while retrieving data from two or three different places. The demonstrated task is drafting the email; it does not establish that an email was sent. The explanation that follows focuses on three design choices behind this kind of retrieval.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give different sources the same query semantics
The first choice is a unified query language for structured data, unstructured data, and APIs. Instead of making the LLM learn a different access vocabulary for every system, give it a common way to express what it needs. Gopal’s rationale is that models already know SQL, whereas the meaning of an application’s custom API may be unclear even to its developers.
His example is a simple filter:
sql
SELECT *
FROM X
WHERE ID > 1;
The operator carries defined semantics: return rows whose ID is strictly greater than 1. With an unfamiliar URL parameter, the model must discover whether a superficially similar filter means greater than, greater than or equal to, or something that only works with Boolean values.
| Interface | What the model must interpret |
|---|---|
SQL predicate ID > 1 | A defined comparison operator |
| Custom API parameter | The application’s particular parameter contract |
A shared language reduces the number of distinct query conventions the model must navigate. It does not, by itself, define business terms such as “top customer”; those still depend on the data and business logic exposed through the tool.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let Python carry out the retrieval plan
The third choice is to let the LLM work out the data-access plan instead of hard-coding a plan for each request. The obvious objection is reliability: if a model struggles to count the Rs in strawberry, why trust it to retrieve and assemble information from several different places?
Gopal turns the challenge back on the audience: count the Is in supercalifragilisticexpialidocious. A difficult unaided counting task is a poor way to use either a person or a model when a computational tool is available. His instruction is: “Don't be mean to the LLM. Set it up for success.”
Ask the model to write Python that solves the problem. For the audience’s counting question, the operation can be expressed directly:
python
word = "supercalifragilisticexpialidocious"
letter = "i"
count = word.casefold().count(letter.casefold())
print(count)
The same division of labor motivates the retrieval design: the LLM produces a program, and code execution performs the operations needed to fetch the data. In the Blockbuster example, that means working out how to obtain the customer and movie information required for the draft, rather than relying on the model to produce those facts unaided. Gopal says they ask their LLMs to run Python to fetch the required data; he presents no quantitative evaluation of planning reliability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bring the data within reach
The closing joke turns the AI singularity into a data singularity: put the data together so AI can access it. For teams building AI, data access is part of the application; for teams managing data, the opportunity is to make it usable by AI. Gopal closes by inviting attendees to the Hasura booth and describing the offering as out in the open. The launch is identified in the event’s historical schedule as Pacha DDN. The practical ambition is the one behind the customer email: let an ordinary request reach the live information and business logic needed to answer it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
- Hasura DDN beta announcementArticle
Tanmai Gopal explains DDN's architecture and composable access to databases, business logic, and APIs.
A technical walkthrough of Hasura's Postgres streaming APIs, including authorization rules based on data and session properties.
A separate Pacha demonstration discusses generated Python and SQL, investigation of empty results, and repair of an incorrect join.
Full transcript
Loading transcript…