AI Engineer World's Fair 2026

Local Models: Trust, Control, Optimization

Loading slides…

Read the talk

Local Models: Trust, Control, Optimization

Open models let builders inspect what they run, specialize it for a product, retain training traces, and optimize the cost of useful outcomes.

From a talk by Carter Abdallah, Vincent Weisser, Lucas Atkins and Chris Alexiuk

Before you start: Familiarity with language-model APIs, inference, and the basic distinction between pretraining and fine-tuning will help.

Why rent a general model for a narrow task?

Why send a narrow enterprise task through an expensive, general-purpose API when the business could own a model specialized for that work? Answering that question requires more than downloadable weights: local and sovereign AI also depend on hardware and accessible training infrastructure. Those layers bring together the participants in Carter Abdallah’s panel: Vincent Weisser, introduced as Prime Intellect’s founder and CEO; Lucas Atkins, Arcee AI’s CTO; and Chris Alexiuk, a product research engineer supporting NVIDIA’s Nemotron family.

Four panelists sit on tall stools, with audience members visible along the bottom of the frame.Open full source frame
Four panelists seated onstage before blue curtains.

For Weisser, Prime Intellect starts with access to the full training stack, alongside access to frontier models. Its work includes helping Arcee train open models and collaborating with NVIDIA through the Nemotron coalition. He tentatively places Nemotron and Arcee’s Trinity among the strongest open models outside China, immediately asking for that assessment to be checked; Atkins’s response is a marketing joke, not a benchmark.

Arcee’s path began with the narrower enterprise problem. Founded in early 2023, it adapted open models for customers and released tooling along the way. Atkins recalls closed APIs costing $100 per million output tokens for tasks that did not need such broad capability, without identifying a particular model or provider. As the company worked with enterprises, customers’ concerns about model origin became another constraint. Atkins expresses respect for Chinese models and their builders while describing the geopolitical concerns those customers brought into purchasing decisions.

That combination pushed Arcee from customization into pretraining. Atkins says Arcee reoriented the company to pretrain a 400-billion-parameter model in six months, working with NVIDIA and Prime Intellect. The resulting goal was a permissive Western model that customers could own, customize, and run wherever they chose.

Alexiuk extends openness across weights, data, training methodology, and training frameworks. Releasing those pieces allows other teams to build on the research rather than starting over. Nemotron pairs that ambition with speed: its design decisions emphasize how quickly the model can generate tokens and whether it works well on hardware outside large, distant data centers. His formulation, “faster models are smarter models,” makes efficient inference part of useful capability.

0:180:36
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:18 · section reference included

Trust starts with knowing what runs

Abdallah turns from collaboration to a harder question: how should builders respond to claims that open models are chaotic or untrustworthy? Atkins first separates trust from safety. A model’s inspectability, its behavior, and the reliability of access to it are different properties, even when public discussion groups them together.

Open weights expose the model’s matrices, and open implementations expose the code used to train or serve it. Atkins points to Prime-RL, vLLM, SGLang, and provider implementations as concrete places to inspect the system. That visibility gives an operator more control over what runs than an arbitrary remote endpoint does. It does not make generated code safe: Atkins explicitly calls for judgment, safeguards, and scrutiny of outputs from a stochastic system, whether the model is open or closed.

Continued access supplies a second meaning of trust. Atkins connects restrictions on Anthropic’s Fable to enterprises becoming more willing to use Chinese open models because they could retain access to the weights. Anthropic’s account of the Fable interruption documents the suspension and subsequent restoration; the migration of customers is Atkins’s account. His strongest assurance about self-hosting or working with a validating partner is best understood as confidence in which model is being served, alongside his earlier warning that its outputs still require scrutiny.

Publishing training material adds another kind of visibility. The discussion points to NVIDIA’s dataset and environment releases, including descriptions of how training tokens are allocated across datasets. The spreadsheet example is illustrative, rather than an audited token budget for a named model. Although the panel also suggests that much about training data can be recovered from weights alone, it provides no method for establishing complete provenance that way. Published datasets offer a more direct inspection path: browse the NVIDIA, Prime Intellect, or Arcee dataset pages on Hugging Face and examine what an actual training record looks like. The panel favors that disclosure without arguing that every release must include it.

6:116:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:11 · section reference included

Post-training changes the cost of a useful result

Control becomes practical when builders can change the model for their own use case. Weisser’s open stack spans pretraining, mid-training, and post-training, but Prime Intellect has concentrated on reinforcement learning and the infrastructure around it. A domain-specific evaluation environment gives the team a target to improve against; post-training provides an economically accessible way to adapt an existing model such as Trinity or Nemotron. The objective is a specialist that combines task quality, speed, and cost more effectively than a general endpoint.

Weisser says Ramp specialized an older model in one or two weeks, beating Opus on its finance task at a fraction of Haiku’s cost. The relevant Ramp case study concerns financial spreadsheet retrieval, so this is a claim about a bounded workflow, not finance automation in general. The panel supplies neither model versions nor a benchmark definition or dollar-cost denominator for its Opus and Haiku comparison. What matters to the proposed strategy is the shape of the result: optimize the task the enterprise actually performs, then evaluate quality and operating cost together.

Atkins connects specialization to budget predictability. A lower token price does not guarantee a cheaper session when an agent consumes more tokens, and model deprecations or more expensive replacements can change a product’s economics. His specific GPT-4–to–GPT-4.5 example does not demonstrate falling prices: the GPT-4 launch and GPT-4.5 API announcement price the latter higher. The broader distinction still holds: builders need to understand the cost of the whole interaction, not just the price printed beside a token unit. Owning the deployment, or arranging it with a partner, gives them more control over those assumptions.

Weisser calls the objective “outcome maxing”: spending should produce more value than it consumes. He invokes Jevons paradox to explain why efficiency can expand demand rather than merely shrink bills. If an agent becomes cheap enough to solve a previously uneconomic task, the business may run far more of those tasks. Optimize useful outcomes per unit of compute, then expect usage to respond.

11:4012:05
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:40 · section reference included

Train the model for the harness

The next layer of specialization is the relationship between the model, the harness, and the product. The harness supplies the tools and execution loop in which the model acts. Atkins recalls OpenAI’s Deep Research as his first compelling agentic experience: he credits reinforcement learning on o3, combined with test-time compute, for sustained research that went beyond repeatedly calling search. From there, he points to product-specific model development at OpenAI, Anthropic, and Google, using GPT-5 and the Atlas browser as his example. His engineering question is why an application builder should assume that an off-the-shelf model is already the right fit for a particular product.

Alexiuk describes the tension from a model developer’s side. A general release must work across harnesses such as Pi and Hermes, while an individual product normally uses one environment. Nous Research, for example, could post-train an open model specifically for its own harness. The phrase “mismanaged genius” captures capability left unused when the model and its operating environment fit poorly.

Customization layerWhat the builder changes
Prompts and skillsInstructions and reusable procedures
HarnessTools and the execution environment
Model post-trainingLearned behavior inside that environment

Closed models still allow substantial prompt and skill customization. Open weights add the ability to train behavior that the surrounding harness cannot reliably supply alone. Alexiuk describes bringing a coding or agent environment into a training workflow; Weisser turns that into a concrete starting point for a Claude Code, Cursor, or Perplexity-style product: choose a strong open model, then post-train it on the harness the product will actually use.

15:5716:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:57 · section reference included

Keep the traces that make the next model better

Atkins compares model usage data to the data collected by cloud and social platforms. Interaction data can improve a model directly through training, or indirectly by revealing which examples the provider should collect or generate. He raises restrictions on training from closed-model outputs as a reason to examine the rights attached to a workflow. Those restrictions depend on the applicable provider agreement; the practical question is whether the product can retain and reuse its own interaction history.

With an appropriately licensed open model, saved harness traces can support two different improvement paths:

  • Fine-tune a smaller model. Use suitable examples from the existing workflow to teach a cheaper model the behavior the product needs.
  • Build better environments. Analyze traces for recurring failures, then turn those signals into tasks and feedback using tools such as Verifiers, NeMo RL, or NeMo Gym.

The trace is useful because it connects the request, the model’s actions, the tools it used, and the resulting outcome. Owning the intelligence therefore also means preserving the data needed to improve it.

Alexiuk brings that loop back to licensing. He says Nemotron and Trinity adopted OpenMDW to make permission to train from outputs explicit. The license’s output permissions do not remove applicable-law or third-party-rights obligations, and the license attached to the particular release remains the one to inspect. The underlying requirement is straightforward: a model-improvement loop needs clear reuse rights as well as working training software.

19:3619:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

19:36 · section reference included

Close the loop with the task’s real environment

Abdallah challenges the assumption that sovereignty necessarily requires giving up performance. A model trained against the right tools and use case may outperform a more general model on that workflow. Browser and computer use make the question tangible: how does a model become effective at operating the systems where work actually happens?

Weisser proposes a sequence:

  1. Build an RL environment that represents the task and provides useful feedback.
  2. Train the model in that environment, so its behavior fits the work.
  3. Deploy to actual users, exposing the agent to the context it must ultimately handle.
  4. Learn from production traces, using the results to improve the next training cycle.

His analogy is the last mile toward driving autonomy: a chatbot does not become a self-driving system without training and deployment in that context. The same logic applies to a bank’s financial agent. His example of deployment to a million accounts is hypothetical, illustrating how a specialized system could encounter the real distribution of user needs.

A small trace record makes the feedback boundary concrete. For an illustrative spreadsheet agent, retain the question, the tool call, the proposed answer, and the verifier’s result separately. The following TypeScript prepares only verifier-accepted examples for later review; it does not update a model:

typescript

type Trace = {
  id: string;
  question: string;
  toolCalls: Array<{ name: string; arguments: Record<string, string> }>;
  proposedAnswer: string;
  verification: "passed" | "failed" | "pending";
};

const traces: Trace[] = [{
  id: "finance-001",
  question: "What was travel spend in March?",
  toolCalls: [{
    name: "search_sheet",
    arguments: { category: "travel", month: "March" }
  }],
  proposedAnswer: "$1,240",
  verification: "pending"
}];

const candidatesForReview = traces
  .filter(trace => trace.verification === "passed")
  .map(trace => ({
    sourceTraceId: trace.id,
    input: trace.question,
    actions: trace.toolCalls,
    answer: trace.proposedAnswer
  }));

The pending answer remains in the trace history but contributes no accepted training candidate. That distinction matters when operational behavior becomes training material. Weisser describes computer-use work with AI-native companies drawing on large collections of production traces, and sees the resulting cycle of specialized models and proprietary feedback as an opportunity for application startups.

22:3522:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

22:35 · section reference included

Specialize capability and share the optimization work

Efficiency also improves below the application layer. Weisser describes collaboration among Prime Intellect, NVIDIA, and vLLM on training and inference for GLM, Trinity, and Nemotron. Shared optimization can lower the cost of running and adapting those models across the ecosystem. A closed provider may make similar improvements without passing all of the savings through to customers; open deployments give operators more direct access to the improved implementation.

Alexiuk informally estimates that most people do not need frontier intelligence for 90% of their tasks. His practical point is about deliberate specialization: a product may need one or two capabilities to work extremely well and have little use for the model’s chemistry knowledge or breadth elsewhere. A general endpoint has to accommodate many users and tasks. A model serving one defined workflow can accept losses outside that workflow in exchange for gains where they matter.

Resource-constrained deployments create another source of improvement. Engineers trying to run models locally have reasons to reduce memory use, increase speed, and remove overhead. Alexiuk compares the accumulation of those contributions to Linux’s role in internet and network infrastructure: many small improvements, made for particular environments, can become shared infrastructure. His argument is that closing the implementation excludes contributors who might find the next useful optimization.

Atkins then qualifies the open-versus-closed framing. He uses closed frontier models nearly every day and regards their work as beneficial. Owned models can become a Linux-like infrastructure layer beneath enterprises, hyperscalers, neoclouds, and services such as Fireworks, Together, Baseten, and Modal. Convenient consumer products can coexist with that layer, just as Macs and Windows coexist with Linux infrastructure. For email, rewriting, and grammar help, a subscription product may be the easiest choice. The opportunity for builders is access to another layer they can engineer, not an obligation to reject ChatGPT or Claude. Abdallah expects both open and closed systems to keep gaining usage.

26:1426:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

26:14 · section reference included

The next thresholds: knowledge work, access, and local hardware

For the closing predictions, Abdallah sets AI Engineer World’s Fair 2027 as the horizon. Weisser expects open models to remain close to the general frontier and potentially narrow the gap if progress in closed models slows. The product change he anticipates is broader adoption of domain-specific knowledge-work agents and computer-use agents, following the adoption pattern already visible in coding.

Weisser’s forecast depends on capability thresholds: a product can exist before its underlying model is good enough to make the experience compelling. He uses Cursor’s growth alongside stronger Opus coding capability as an example, then describes GLM-5.2 as a perceived open-model inflection. These are his interpretations of product momentum, not measured causal comparisons. His expectation is that stronger open models will unlock many more such businesses over the following year or two.

After joking about a trillion-dollar combined valuation for Prime Intellect and Arcee, Atkins shifts to distribution. He raises reported Fable and GPT-5.6 access restrictions as signs that capability access could become more politicized, and argues that sovereign intelligence will matter more as a result. His extremely small estimate of present local-model adoption is rhetorical rather than a measured baseline. Atkins hopes that, by the following year, 10–15% of people who have ever used AI will have run a model locally. He frames that as an advocacy goal as much as a prediction: open science and public discussion should have a place in decisions about who gets access to powerful systems.

Alexiuk similarly sees the coming year as consequential for open intelligence, then offers several connected forecasts:

  • Everyday local work. A sufficiently capable model on a MacBook could handle most routine AI tasks without a remote API within a year.
  • Systems of specialists. Multiple specialized models or swarms could become more important than reliance on one universal model.
  • Architectures for household hardware. Text diffusion and other architectural changes could make models better suited to local devices.
  • Agent operating systems. Computers could arrive with agent environments preinstalled, illustrated by a Spark preloaded with Hermes.

Together, these predictions describe changes in the model, the runtime, and the way a user first encounters the system.

32:2132:29
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

32:21 · section reference included

Make local intelligence something people can use

Atkins expects consumer exposure to arrive through familiar devices. He predicts that a September iPhone and Siri release will prompt questions from family members, much as DeepSeek did. Weisser connects that possibility to two developments arriving together: open models becoming sufficiently capable and device hardware becoming strong enough to serve them at useful speeds. His scenario of Opus-level capability on a phone or laptop is a forecast for the following year or two, with a stronger fit for consumers than for the heaviest enterprise agents. It could give application developers a new platform: intelligence supplied by the user’s own device.

Atkins says a four-billion-parameter phone model can already be more useful than launch-era GPT-4, without naming a model, device, or task benchmark. He uses the comparison to emphasize deploying existing capability well, alongside pursuing new capabilities in drug discovery and scientific exploration. The practical opportunity is not limited to whatever model currently leads the frontier.

Abdallah closes by tracing a change in public experience: AI moved from a research concern into everyday awareness through ChatGPT, while open models are only beginning to reach people who have never tried them directly. Later panels will examine local systems and their relationships with frontier models, but realizing this next shift also requires builders to create the first useful experience of owned intelligence. Alexiuk extends that responsibility beyond the engineers in the room to their friends and communities. Open access becomes durable when people can use it, improve it, and explain why it matters.

39:4439:59
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

39:44 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…