AI Engineer Summit 2025
Beyond the Consensus: Navigating AI’s Frontier in 2025
Loading slides…
Read the talk
Beyond the Consensus: Navigating AI’s Frontier in 2025
Grace Isford traces the gap between model progress and dependable agents, using a constrained flight search to explain cumulative errors, personalized evaluations, and better product design.
From a talk by Grace Isford
How do you navigate a frontier that keeps moving?
How do you decide what to build when the AI frontier moves faster than the consensus about it? At the AI Engineer Summit 2025, Grace Isford approaches that question as a partner at Lux Capital, investing in technologies before their usefulness is widely understood. The portfolio examples establish the breadth of that perspective: Hugging Face provides a GitHub-like home for machine learning; Together AI supplies an open-source AI cloud; Physical Intelligence develops a robotics software brain; and Tokyo-based Sakana AI explores nature-inspired evolutionary algorithms. Isford also points to Sakana’s AI CUDA Engineer, which she says launched the preceding night.
New York is part of this investment thesis, not simply the conference location. Isford says Lux was founded there, made its first AI investment there in 2013, and has a majority of its AI portfolio headquartered in the city or operating a major hub there. The combination of research and engineering talent with Fortune 500 customers creates an opportunity to connect frontier capabilities with organizations that can use them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
More models, more reasoning, more infrastructure
Starting with Stable Diffusion in August 2022, Isford’s timeline depicts a rapidly steepening release curve. She describes acceleration across the following two and a half years, especially the preceding eighteen months. Development is also spreading beyond OpenAI and Anthropic to xAI, Mistral, and DeepSeek. The Grok launch is one recent example of a broader shift: more organizations are producing capable models while improving computational efficiency.
Early 2025 adds infrastructure commitments and public attention to that technical momentum. The Stargate announcement proposes $500 billion of investment over four years; it is a plan, not completed expenditure. Although Isford groups the US government with the participating companies, the announcement distinguishes government support from the initial corporate equity funders: SoftBank, OpenAI, Oracle, and MGX. She also highlights OpenAI o3 and its ARC-AGI preview result. Her description of exceeding human performance needs a narrower reading: this was a benchmark-specific preview evaluation with substantial test-time compute, which ARC Prize characterized as approaching human-level performance, not establishing general human superiority. The subsequently released o3 was a different system from that tested preview.
Next comes DeepSeek R1. Isford recounts its launch, the accompanying decline in NVIDIA shares, and DeepSeek’s rise to the top of the App Store. The France AI Summit and Macron’s initiative complete her picture of renewed international participation: the frontier is expanding across model developers, infrastructure providers, and governments.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Favorable conditions do not establish autonomy
These developments create what Isford calls a perfect storm for agents. Reasoning models—OpenAI o1, o3, R1, and Grok’s latest reasoning model—suggest new capabilities, including performance she describes as surpassing humans on some tasks. Test-time compute is a central mechanism: a system spends additional computation during inference rather than only during training, giving it more opportunity to work through a problem before answering.
Engineering efficiency supplies another part of the argument. Isford praises DeepSeek’s hardware and engineering work without resolving the dispute over its training cost. Cheaper inference and hardware make more agent activity economically plausible, while DeepSeek and Llama narrow the gap between open and closed models. Data-center investment in the United States, Europe, and Japan—including efforts involving SoftBank and NVIDIA—provides additional capacity.
Yet the conditions for agents are not the same as agents that reliably work. Isford sees momentum without the decisive breakthrough. Her definition sets a demanding bar: an agent is a fully autonomous system in which LLMs direct their own actions. That definition turns attention from the quality of an individual response to whether a system can carry an entire task through successfully.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An ordinary flight request becomes a constraint problem
Consider booking a Monday flight from New York to San Francisco. Isford tries this with OpenAI Operator, using its early-2025 browser-based research preview. That historical experience precedes Operator’s later integration into ChatGPT agent. What sounds like a simple booking instruction contains several interacting requirements.
| Constraint | Requested outcome |
|---|---|
| Departure | After 3 p.m. Monday; avoid rush-hour traffic |
| Airline | United, JetBlue, or American |
| Upgrade | Preserve an opportunity to upgrade from economy |
| Budget | Below $500 under the work expense policy |
| Seat | Aisle, not too close to the bathroom |
| Arrival | Before midnight |
The agent must satisfy the combination, not merely locate an available flight. A departure time can meet the stated time window while still making the trip to the airport impractical.
On the first attempt, Operator goes to Kayak and finds no flight. Isford says it does not even appear to look for United or American. The slide’s completed annotation makes those two failures explicit: the search misses preferred airlines and produces no result.
On the second attempt, Operator uses Skyscanner and finds a flight departing JFK at 5:30 p.m. That result conflicts with the request to avoid rush-hour traffic. Isford also cannot select her seat. The search has progressed from no result to a result, but it still has not completed the requested task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Small failures extend beyond hallucinations
Hallucinations and fabricated answers receive plenty of attention, but this booking attempt exposes a wider problem: small errors accumulate across decisions, integrations, practical judgment, and preferences. Isford offers a non-exhaustive taxonomy. Decision error means choosing the wrong fact—for example, interpreting San Francisco as the city in Peru rather than California. Overthinking or exaggeration can also push a decision off course. The wrong destination is a hypothetical illustration, not an outcome of her demonstrated search.
Implementation error concerns access and integration. During the Skyscanner attempt, Isford has to complete a CAPTCHA, interrupting the flow. Operator’s original design explicitly required human takeover for CAPTCHAs, making this a boundary of its autonomy rather than necessarily a malfunction. Losing access to a critical database would create a related operational problem: the agent could know what it should do but lack the access needed to continue.
Heuristic error means applying the wrong practical criteria. The flight search does not allow adequately for reaching JFK, and it never asks whether Isford is starting from Manhattan, Brooklyn, or somewhere else. Without that information, a seemingly acceptable departure time can still be a bad recommendation.
Taste error means missing a personal preference. Isford does not like flying on Boeing 737 MAX aircraft, but she did not include that constraint in the original prompt. An agent could satisfy every explicit requirement and still offer a flight she would refuse to board. This is a different problem from misreading an instruction: the relevant information was never supplied.
A fifth issue is the perfection paradox. Capabilities that initially seem extraordinary quickly become the baseline for expectations. Users grow impatient when o1 thinks too long or Operator moves at human speed. Even getting the first attempt right is insufficient if subsequent attempts are inconsistent. Reliability, latency, and the user’s expectations all shape whether the experience feels successful.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Fifty steps change what “accurate” means
Isford illustrates the cumulative effect with two agents that begin at 99% and 95% accuracy per step. Both sound strong in isolation. Across fifty consecutive steps, however, the chance of completing every step correctly can fall sharply.
One mathematical interpretation assumes constant, independent per-step success and requires every step to succeed:
Under those assumptions, the 99% agent succeeds across all fifty steps about 60.5% of the time, versus 7.7% for the 95% agent—a gap of about 52.8 percentage points. This makes precise the rough separation Isford describes; the recording does not specify the probability assumptions or present an empirical trial.
The engineering target is therefore consistent completion of the whole task, not simply a high rate of correct individual actions. Booking a flight combines enough decisions and dependencies to expose this difference. Multi-step, multi-agent systems can amplify it further. Isford’s response is five strategies: curate the data, evaluate the actual outcome, contain failures with scaffolding, understand the workflow, and build richer multimodal experiences.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Curate what the agent needs—and what it learns
The first strategy is data curation. An agent’s inputs are messy, unstructured, and scattered across silos. They extend beyond web pages and text to designs, images, video, audio, warehouse sensors, and data generated by the agent itself. Curation must cover proprietary source material, agent outputs, and the workflow data used for quality control.
That work should continue after launch. Isford proposes designing a data feedback loop from the beginning, so product usage can improve the system rather than disappear into an activity log. For the travel agent, the initial curated dataset would contain her airline preferences and avoidance of the 737 MAX. Subsequent bookings would supply additional evidence about what she accepts, rejects, or changes. Feeding that information back into the preference dataset could make future recommendations more personal. It is a proposed improvement to the booking system, not something her Operator attempt already accomplished.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluate the outcome for the person using it
The second strategy is evaluation: collect model responses and determine what counts as correct. In verifiable tasks, a clear answer supplies a strong signal. Isford points to Grok 3 math and science benchmarks as examples of this style of evaluation. But whether she will like a particular airplane seat is not reducible to the same kind of answer key. That requires signals about her preferences.
The deep research products from OpenAI, Perplexity, and Gemini illustrate why the task and the user matter. A product that works well for everyday research may not be the best fit for venture-capital market research or scientific and academic work. Evaluation must capture those differences rather than assume a single ordering applies to everyone. Collecting human preferences makes it possible to assess the outcome against the user’s intended purpose. Sometimes directly trying an agent on that purpose reveals more than a leaderboard number does.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Stop a local error from becoming a system failure
The third strategy is scaffolding: the surrounding system should prevent one failure from cascading into others. Isford cites Lux portfolio company Ramp, reporting that its infrastructure logic contains failures in new applied-AI features so they do not propagate across the agentic system or production infrastructure. The key property is containment; she does not specify Ramp’s implementation.
A compound system can coordinate components and bring a human back into the loop when necessary. Stronger reasoning agents introduce further possibilities: an agent might recognize a mistake and correct its path, or recognize uncertainty and interrupt execution before continuing. In the travel example, a checkpoint could verify the airport timing and redirect the booking agent before an unsuitable flight becomes a completed purchase. The scaffold supplies an opportunity to recover instead of allowing each earlier choice to dictate the next one.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Shared models make workflow understanding more valuable
The fourth strategy moves from reliability to user experience. Isford views foundation models as rapidly depreciating assets: when competing applications can access the same underlying capabilities, model access alone offers little durable differentiation. A GPT wrapper can still be a valuable product if it deeply understands the user’s workflow and makes human-machine collaboration effective.
Her examples identify different ways that understanding appears in a product:
- Clarify the objective: Deep research asks questions before proceeding, establishing what the user is actually trying to accomplish.
- Anticipate the next action: Windsurf from Codeium reflects the developer’s workflow and helps predict the next step.
- Fit the existing environment: Harvey integrates into legacy legal systems, which Isford presents as a route to practical value for lawyers.
These are interaction and integration decisions, not merely differences in the text a model generates.
The same distinction applies to coding, customer support, and sales: applications using shared models compete through product quality and UX. Isford is especially interested in domains where proprietary data meets specialized work—robotics, hardware, defense, manufacturing, and life sciences. Understanding how a biologist, defense contractor, or chemist works can determine whether an AI capability becomes a useful product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build beyond the chatbot
The fifth strategy is to build multimodally, not merely combine multiple models. Isford’s ambition is a dramatically more personalized experience, rather than a measured improvement claimed for an existing system. The chatbot is only one possible interface. Vision, hearing, and increasingly natural voice can give agents other ways to receive context and interact with people.
She extends the idea beyond familiar media. Lux investment Osmo works on digitizing smell. Touch and robotics raise questions about embodiment, while memory could let an assistant know a person more deeply over time. Together, these directions change both the information available to an agent and how a person might experience its help.
Isford suggests that a sufficiently novel product experience can exceed expectations even while the underlying agent remains inconsistent. Her concrete example is Lux portfolio company tldraw: a visual canvas where AI can be expressed through brushstrokes. She describes tldraw Computer as combining models in tandem without requiring the user to think about the large language model operating underneath. The visible canvas of connected boxes and drawn strokes illustrates an interface organized around visual work instead of a chat transcript.
The favorable conditions for agents are real, but dependable autonomy will not arrive overnight. Wrong answers, missing preferences, unsuitable criteria, and disappointed expectations accumulate. Data curation, evaluations, and scaffolding address those failures; UX and multimodality determine what the resulting product enables people to do. Isford’s closing challenge is to pursue both: make the system more dependable while imagining a better way for the user to work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Operator's original research-preview announcement explains browser interaction, human takeover and safety limits.
The original announcement sets out the proposed investment timeline and participating companies.
Further reading
The benchmark organizer reports preview o3's ARC-AGI-1 results, compute conditions and limitations.
Osmo describes its laboratory process for measuring scent molecules and reconstructing a fragrance.
Updates since the talk
A September 2025 account of robust-kbench and an agentic workflow for generating, verifying and optimizing CUDA kernels.
Full transcript
Loading transcript…