AI Engineer World's Fair 2026
Enterprise Agents Have a Structure Problem - Ishita Daga, Tesla
Loading slides…
Read the talk
Enterprise Agents Have a Structure Problem
Better data-agent answers depend on knowing which source to trust, keeping definitions current, and choosing the metric the requesting team actually means.
From a talk by Ishita Daga
Before you start: Familiarity with SQL queries, business metrics, and agents that retrieve organizational data will help.
A bad answer does not always call for a bigger model
A data agent gives a bad answer. Should you give it a bigger model, a longer context window, or more knowledge? The usual additions—Markdown files, documents, MCP servers, and plugins—can all help. But adding capacity and information does not tell the agent which information deserves authority. Ishita Daga’s starting point is that better answers require structure around the model, not just more material inside its context.
Three different failures can hide behind the same incorrect answer:
- Ambiguity: The agent does not know which table, column, data source, or knowledge base to use. Several sources may contain relevant information without identifying which is authoritative, which is cleanest, and which allows more flexibility.
- Staleness: Decisions, KPI definitions, and processes change, while the context supplied to the agent remains unchanged.
- Preference: An individual or team expects a particular metric definition, query, or set of filters—even when a canonical query already exists.
These failures need different remedies. Source selection needs an ordering; changing knowledge needs a lifecycle; team preferences need a way to connect the requester with the intended calculation. Daga treats that last problem as especially open-ended.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with the most curated source
An agent should not weight every knowledge base equally. Daga proposes a source-of-truth hierarchy that begins with the cleanest, least flexible source and moves toward more dynamic sources as the question requires. This makes authority part of the retrieval decision rather than leaving the model to reconcile an undifferentiated collection of documents and schemas.
The first layer is a semantic layer: a curated collection of queries, KPI definitions, metric calculations, and business meanings. The agent finds the closest relevant KPI and uses its associated definition and query information to answer. This constrains interpretation before query generation begins.
The second layer is a set of canonical, parameterized queries. Daga initially calls this layer canonical tables, then describes reusable query patterns. The agent chooses an appropriate pattern and has more latitude to adjust parameters, filters, or the query itself. That flexibility lets it answer questions beyond the exact metrics already represented in the semantic layer.
The third layer is a database graph. Tables connect to their columns and to the metrics they can answer, with relationships available in both directions. An agent can use these connections to find the data needed for a question and construct a query over it. The expanded reach comes with substantial work: the graph must first be built, then maintained as tables, columns, and definitions change.
| Layer | What the agent uses | Main tradeoff |
|---|---|---|
| Semantic layer | Curated KPI definitions and queries | Strong constraints, narrow flexibility |
| Canonical queries | Parameterized query patterns | More freedom over queries and filters |
| Database graph | Metric–table–column relationships | Broad flexibility, difficult maintenance |
The hierarchy is an order of approach: use the curated definition when it fits, then move toward sources that allow more construction and interpretation.
Daga estimates that the semantic layer and canonical queries can solve about 80% of problems, with the database graph addressing the remaining 20%. This is a prioritization estimate, not a reported accuracy benchmark. Her implementation recommendation follows from the maintenance tradeoff: build the first two layers first, then add the graph for questions they cannot cover.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give context a maintenance loop
Even a well-ordered source hierarchy can give confidently outdated answers. Processes change, definitions are deprecated, and manually maintained .md files or skills become difficult to keep current. Daga’s response is a context lifecycle with two components: maintained knowledge sources and a feedback loop.
A live source, in this framework, is one that people regularly update, review, and curate. Examples include GitHub, CRM tools, Tableau, and the dbt Semantic Layer. The useful property is not simply that the agent has a connector. It is that the connected source has an ongoing process for keeping its information current.
The feedback loop begins with corrections that might otherwise disappear into a conversation. A user may report that the database is wrong, a definition is outdated, a metric has a new calculation, or a query needs a different filter. Each of these is an event worth capturing. Logging makes the correction available for a context update rather than limiting its effect to one answer.
Those events also need evaluation. Daga describes two complementary approaches:
- Human evaluation: Curate questions with human annotations or have people assess the agent’s answers.
- Automated evaluation: Revisit questions asked over the last few days and compare newly generated answers with their actual answers.
The recurring process is to log corrections, evaluate behavior, and update context. Without evaluation, a team cannot see whether its agent is improving or how its performance changes over time. Refreshing context becomes useful operational work when the team can assess what the refresh did to the answers.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One milestone question, two valid calculations
Preference creates a harder problem: the source can be authoritative and current, yet still represent the wrong interpretation for the requester. Different teams may calculate the same named metric differently or apply different filters to the same query. Daga illustrates this with two teams asking for the average time taken for a milestone.
Team A measures from completion of the previous milestone to completion of the current milestone. Team B measures from the start of the current milestone to the start of the next milestone. Both are valid in the example, but they measure different intervals.
| Team | Interval | Slide result |
|---|---|---|
| A | Previous completion → current completion | 18.4 days |
| B | Current start → next start | 23.1 days |
The slide’s values illustrate the consequence of choosing different endpoints; they are not an agent-performance comparison.
The distinction can be made explicit in SQL. For an illustrative PostgreSQL table named milestone_instances, with one row per milestone instance and timestamp columns for the relevant endpoints, the two definitions become separate expressions:
sql
SELECT
AVG(
EXTRACT(EPOCH FROM (current_completed_at - previous_completed_at))
/ 86400.0
) AS team_a_average_days,
AVG(
EXTRACT(EPOCH FROM (next_started_at - current_started_at))
/ 86400.0
) AS team_b_average_days
FROM milestone_instances;
Naming both calculations makes their difference inspectable. It does not tell the agent which one a user intends when asking for average milestone time. The missing input is the requester’s intended definition, not another timestamp or a larger context window. Daga sees resolving that intent as an open problem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Connect the requester to the right definition
One approach Daga is trying is to store every alternative calculation in the semantic layer. The user can then explicitly tell the agent which definition to use. For the milestone example, that would mean selecting either the completion-to-completion calculation or the start-to-start calculation. The catalog can represent both meanings, but explicit selection does not itself preserve the user’s preference. On the next underspecified question, the agent faces ambiguity again.
The other approach is agent memory, such as Mem0 or a memory.md file. Daga’s concern is that storing a preference is not the same as selecting the correct metric in context. A memory can retain what a user prefers without establishing the distinction between two business definitions or when each should apply. Her critique concerns that selection step, not the absence of user-specific memory.
The desired behavior is to route the agent to the appropriate metric based on the individual or team using it. In the milestone example, the shared agent would need to connect the requester with the relevant interval definition rather than repeatedly require a fully specified question. Daga presents this as a research direction, not a demonstrated routing implementation.
This extends the engineering task beyond organizing knowledge. An enterprise agent needs a structure for choosing sources, a process for maintaining and evaluating context, and a way to incorporate individual preferences into shared behavior. Daga’s closing aspiration is a shared data agent that can retain those individual meanings without losing the organization’s common knowledge. The difficult part is making the same agent answer appropriately for different people using the same words.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Documentation for defining shared business metrics, handling joins through MetricFlow, and deploying metrics to downstream tools.
Examples for storing preferences and retrieving memories scoped to users, agents, applications, and sessions.
Further reading
Connection instructions and limitations for consuming semantic models in Tableau, including how local overrides affect inherited definitions.
Full transcript
Loading transcript…