AI Engineer Summit 2023
Building efficient hybrid context query for LLM grounding
Loading slides…
Read the talk
Building efficient hybrid context query for LLM grounding
Natural-language product search must capture what a shopper wants while respecting constraints such as price and category. Simrat Hanspal’s walkthrough connects those retrieval tasks, then tests what happens when a shopper asks the search interface to write to the catalog.
From a talk by Simrat Hanspal
Product search needs meaning and structured constraints
A shopper may know what a product should do and how much they can spend without knowing the catalog’s exact wording. Their request combines a description with a constraint. A product-search application therefore needs to distinguish descriptive intent, conditions on structured fields, and requests containing both. Replacing keyword input with natural language makes the interface more expressive, but the retrieval system still has to translate that expression into appropriate operations.
A language model’s training data is not a live product catalog. Retrieval-Augmented Generation, or RAG, supplies relevant external information alongside the user’s question so the model can ground its response in that information. For product search, this means retrieving catalog context that fits the request, rather than relying on the model’s existing knowledge of products.
Hanspal introduces security before building the pipeline. Useful context must not become a route to sensitive data simply because a user asks for it. That is a requirement for the application, not a guarantee conferred by adding retrieval: the system must control which data the request can reach.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Dynamic requests need bounded data access
Why revisit secure retrieval when data-driven applications have existed for years? Hanspal’s distinction concerns the shape of the application’s data interface. In the conventional product-search application Hanspal describes, the software consumes a relatively fixed feed. Records and results change, but new functionality generally comes from changing the application around that interface.
A context-driven application must instead select information according to an open-ended natural-language request. The required context can vary with what the user asks. That flexibility also gives users room to request behavior the application designer did not intend, so interpreting a request and deciding whether to permit it are separate responsibilities.
Hanspal introduces Hasura’s GraphQL API and permission system as the data-access layer. A unified interface can expose different data sources while allowing requests to vary in structure. GraphQL expresses the operation; permissions govern what the execution role may do. A dynamic query language does not itself establish authorization.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Prepare the records, then expose one GraphQL interface
The retrieval interface must accommodate three kinds of request:
- Semantic search matches the meaning of a request against product descriptions in a vector database.
- Structured search applies conditions to fields such as
priceandcategory. Natural language must become a structured operation, expressed in SQL or GraphQL, for example. - Hybrid search combines descriptive similarity with structured conditions.
Here, hybrid means semantic retrieval plus structured constraints. The demonstration does not introduce a fusion of keyword and vector rankings. Hasura provides one GraphQL interface for all three request shapes.
Hanspal’s Streamlit application supplies the user-facing entry point. It accepts a natural-language request, calls a language model to generate GraphQL, and submits the generated operation to Hasura. The model performs the translation; Hasura executes the resulting data request.
In the console’s Data tab, a Postgres product table and a product vector table are already connected. Although the architecture can expose multiple sources, Hanspal simplifies the live searches to use only Weaviate. Its prepared records hold vectors alongside structured product fields, including price and category. The demonstrated hybrid search therefore does not require a live join between Postgres and Weaviate.
Preparation happens before the search request. Hanspal uses Hasura event triggers to initiate vectorization when a new product is inserted into Postgres, then saves the vectorized record in the vector database. The insertion event starts the ingestion work; subsequent searches operate on the prepared collection. The console’s Product_vectors table shows the product records with their name, description, price, and category fields.
The console’s API tab provides a workspace for executing queries and inspecting their results. Hanspal then moves to Streamlit, whose configuration panel contains the Hasura endpoint, an admin secret, and an OpenAI API key for the Chat Completion API. These settings connect the demonstration’s components. The execution privileges attached to those requests become consequential in the later authorization test.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three shopping requests exercise the retrieval interface
The first request asks for essential oils for relaxation. The generated GraphQL operation identifies that descriptive intent as the input to semantic search, and the application displays returned products. The task is to match what the shopper wants the product for, rather than apply a numeric condition.
Next, Hanspal asks for products costing less than $500. The generated operation identifies price as the field and less-than as the comparison: price < 500. The application displays products satisfying that condition. Unlike the relaxation request, this search depends on a precise field comparison.
The third request combines the two: essential oil diffusers in a price range of $500 to $1,000. The generated GraphQL operation includes a semantic search for diffusers and a price-range filter, and the application again displays results. Semantic relevance identifies the kind of product; the structured constraint determines which prices qualify. Weaviate filters provide the structured part of retrieval from the prepared collection.
Consider an illustrative diffuser record priced at $750. Its description can supply the semantic match, while its price falls inside the requested interval. Neither condition substitutes for the other: an unrelated product at the right price would not satisfy the descriptive request, and a relevant diffuser outside the range would not satisfy the price constraint. The $750 record is a teaching example, not a reported result from the demonstration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An insert request exposes the permission boundary
The successful searches have exercised the intended behavior. Hanspal now turns to an unintended operation: asking the search interface to insert a hair-oil product named special oil, priced at $10,000, in category home, with description Fantastic hair oil and product ID 10001. This is a request to change the catalog, not retrieve context from it.
The model generates a GraphQL insert mutation, and the initial execution privileges allow it to succeed. Hanspal returns to the console and filters the table for the new product ID, removing quotation marks from the lookup value because the identifier field is an integer. The matching record appears: the search application has actually written to the database.
The repair is a permission change. In the console, Hanspal creates a product-search role and grants it only select access. For simplicity, Hanspal applies no row checks and makes all product columns accessible. The role can read products but receives no insert permission. This is an operation-level restriction; the deliberately broad read permissions do not establish protection for sensitive rows or columns.
Hanspal retries the insertion request under the restricted role, changing the product ID to 7002. The model still generates an insert mutation. This time execution returns an error because the role lacks insert permission. The application displays the role as product_search_bot, and the GraphQL error reports that the insert field cannot be queried on mutation_root. The attempts preserve the insertion intent and operation type, but they are not byte-identical requests.
The rejection separates generation from authorization. The model remains willing to propose a write, but that proposal cannot grant the execution role permission to perform it. The application’s access boundary holds even when the generated operation falls outside the intended search behavior.
Hanspal closes by returning to the hybrid product-search use case: descriptive requests and structured filters can share a retrieval interface while execution permissions constrain its operations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Configure access by table, role and operation—the boundary that stops the generated insert in the demonstration.
Use database events to trigger the ingestion work that keeps the vectorized product records up to date.
Combine numeric and categorical constraints with retrieval from the prepared product collection.
Further reading
- Hasura + Vector DBsArticle
Hanspal's earlier tutorial builds a permission-aware Postgres/Weaviate application and wires database events to vector ingestion.
The companion workshop to Hanspal's vector-database tutorial supplies setup notebooks and event-handler code.
Hanspal extends the retrieval story to secure data access and operational concerns using Hasura and Portkey.
Full transcript
Loading transcript…