AI Engineer Summit 2025
WTF do people use Open Models for??
Loading slides…
Read the talk
What people use open models for
Open-model usage reveals two different markets: people choosing a conversational personality, and businesses protecting reliable workflows from unwanted change.
From a talk by Eugene Cheah
Before you start: Basic familiarity with language models, API requests and production software is helpful; no knowledge of model training is required.
What are all those models doing?
What do people actually do with the flood of open models? Eugene Cheah opens with Hugging Face’s accelerating catalog: he reports more than 50,000 model uploads per month over the preceding year. DeepSeek-R1 supplies the striking example—a model he presents as challenging GPT-4o without a billion-dollar budget. That comparison depends on the task; the original R1 model card shows stronger results on some reasoning benchmarks, not superiority across every evaluation.
Cheah reports more than four million downloads of a 685 GB model in the preceding month. If each represented a complete transfer, that would imply 2.74 exabytes of data, or roughly 690 years over his 1 Gbps connection. The conditional matters: Hugging Face’s download documentation describes counts of designated file requests, including HEAD requests, rather than complete weight transfers. His separate prediction of another 500 uploads during the talk also does not follow from the monthly rate alone. The opening establishes enormous interest, but download counters cannot tell us either the bytes transferred or the work performed.
Cheah’s observation point is Featherless AI, where he is CEO, alongside his role as co-lead of RWKV. At the time of the talk, he describes a service offering unlimited API requests to more than 3,700 open models for thousands of users. The individual plan costs $25 per month, includes R1, and sits alongside larger commercial plans. The ambition is to keep expanding toward the Hugging Face catalog; the resulting mix of customers gives Featherless a view into both personal experimentation and production workloads. These are the service conditions behind the February 2025 observations, not a current pricing guide.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Equal prices reveal preferences—and hide fine-tunes
For individual users during one week in February 2025, R1 leads Featherless’s usage chart, followed by the Llama 3 family, Mistral NeMo 12B and Qwen. Individuals do not pay per token or face different prices for different models. Instead, the plan limits them to one large-model request at a time, while smaller models generally respond faster. That removes a major price incentive from the choice, although latency still matters. Cheah interprets the remaining preferences as a mixture of personality, reputation and feel, rather than simply MMLU scores.
The chart changes meaning when its categories change. Grouping by model family makes Llama and Qwen look like coherent products. Grouping by model name splits those wedges into many fine-tunes, each with different behavior and intended uses. A family’s popularity can conceal demand for very specific personalities. That distinction becomes especially useful when interpreting creative and conversational workloads.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Production changes the chart
Businesses bring a different preference: consistency. Developers want to decide when their model changes, rather than discover that a provider has changed it for them. When Cheah adds the scaled commercial customers excluded from the individual chart, the smaller Mistral NeMo becomes dominant. It is already an older model in a fast-moving market—roughly seven months past its July 2024 release at the February event. Cost helps explain its persistence: Cheah says Featherless provides access to four small models for the price of one large model.
Once deployed, models can remain in place for quarters or years. Cheah credits NeMo’s early appeal to its Apache 2.0 license and its ability to replace GPT-3.5-class workloads, contrasting that license with restrictions in the Llama license that made some enterprise lawyers uncomfortable. He also credits cloud-platform defaults, including AWS and GCP, and hundreds of fine-tuning tutorials with reinforcing adoption. Those distribution paths matter because an example copied into a tutorial can become a dependency in a production system long after the model stops being fashionable.
Consider a narrow application whose prompts and monitoring already produce dependable results. Cheah describes teams that have tuned such systems to above 99% task accuracy; he does not provide a shared evaluation set for that figure. A model change can invalidate the prompts, disturb measured behavior and create another round of repairs. Keeping a fixed model version preserves the baseline against which improvements are evaluated.
The same persistence appears in safeguards. Cheah reports that Llama 2 accounts for about 2% of Featherless’s workload. He highlights Llama Guard, based on Llama 2, as an example: older safeguard tutorials continue to send both existing production systems and newly arriving teams toward it, despite newer alternatives. The age of the model is less decisive than whether the surrounding workflow still works.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From model identity to creative and social uses
Featherless’s strict no-text-logging policy limits what it can directly observe. Instead, the team infers use from model identity and requesting-application metadata—a Llama Guard request is a strong clue about safeguards—and asks customers what they are doing. It also exchanges observations with aggregators such as OpenRouter. Cheah’s approximate ordering is creativity and friendship, coding, ComfyUI-style workflows, RAG and chat interfaces, then other agents and work. Customers include individuals, internal business systems and products that repackage the API in mobile apps, web apps or agents. The percentages that follow mix traffic and request descriptions without a common published denominator, so they describe this observed ecosystem rather than a precise market census.
Cheah estimates that creative writing, roleplay, companionship and the smaller therapy/journaling category together account for 30–40% of traffic. NovelCrafter illustrates the writing workflow: outline a novel, manage its lore, draft scenes, maintain high-level narration and collaborate with both humans and AI. Authors and fan-fiction writers use the model within a continuing creative project, rather than as a one-question answer engine. Adjacent uses include generating Dungeons & Dragons-style game content.
For roleplay and companionship, Cheah names Wyvern Chat and SillyTavern, along with mobile products that call Featherless underneath. He describes this as the largest non-code, non-agentic category by active users. Its association with explicit content obscures other reasons people use it; he connects the demand to recurring references to the film Her in closed-model product discussions too.
Cheah reports that a demographic group whose identity is redacted in the public transcript accounts for more than 60% of users in the companionship segment, without supplying a sampling method. He compares the mismatch between public image and actual demand to romance publishing: a large audience can remain underrepresented in how an industry presents itself. An unnamed app maker describes users having long daily conversations to decompress, particularly when real-life partners are absent or emotionally unavailable. Cheah turns that observation into an interpersonal point about being available to listen, which leads into the neighboring category of therapy and journaling.
That neighboring category is fragmented across many dedicated apps, but the interface need not be specialized. Some surveyed users simply choose a ChatGPT-like client or a therapy character inside a companion app. These are descriptions of how people use the products, not evidence that the products provide effective clinical care. The common requirement is a conversation that helps the user reflect.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The technical requirement behind “vibes”
For these users, MMLU is not the main selection criterion. Community fine-tunes compete on conversational behavior, and favorites can change weekly. The apparently vague word vibes includes a concrete requirement: do not rush to solve the conversation. In the therapy and journaling example, the model should help a person work toward their own answer instead of immediately prescribing what to do. The desired behavior can run against the answer-first habits of instruction-tuned assistants.
Story writing, roleplay and companionship have a related requirement: slow burn. Cheah compares rushing a story to starting Game of Thrones at the first episode and then jumping straight to the finale. The entertainment lies in what happens between those endpoints. A model that resolves every tension immediately can be efficient at answering and poor at sustaining the experience.
Fine-tunes make those preferences concrete:
- Voice: Cheah names alpindale Magnum for a cowboy-style personality.
- Genre: He names Rossini for science fiction, which he says many other models handle poorly.
He expects these particular favorites to change quickly. Yet users also return to old models, treating their distinctive behavior like the remembered charms of a retro game. In this category, replacement does not necessarily make the predecessor worthless.
Cheah puts this broader community, including closed products such as Character.ai, in the tens of millions of users. He contrasts that scale with other categories he describes as not yet reaching millions. These are his broad audience estimates, but they explain why a large catalog of differently behaving models can matter more than a single leaderboard winner.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Coding shifts from completion to intervention
Coding has a smaller human audience but a much larger appetite for tokens per person. Cheah places coding at 20–30% of traffic. He first separates familiar IDE features—autocomplete and chat-based editing—from more agentic workflows. Replit, Mistral and other labs supply small coding models in the 3–12 billion parameter range, and he considers autocomplete essentially solved for this purpose. The competitive frontier has moved toward agents that carry out larger changes.
The useful agent is not necessarily the most autonomous one. Cheah describes developers finding SWE-agent- or Devin-style autonomy excessive when it leads to loops. His preferred pattern asks clarifying questions, accepts corrections and lets a human intervene quickly. Vibe coding takes that interaction far enough that the person may never edit the code directly, instead steering the work through chat.
Cheah estimates that a developer deeply engaged in agentic coding can generate 1,000 times the input and output token traffic of one companion-chat user. Repeated context, tool results and revisions make a comparatively small population expensive to serve. In his account, at most tens of thousands of simultaneous coders can therefore produce a rapidly growing share of overall traffic.
Open models are entering a category still dominated by commercial ones. Cross-referencing OpenRouter, Cheah reports Claude Sonnet’s dominance in agentic coding at a ratio greater than 10:1, without defining the comparison population precisely. Claude Sonnet also benefits from the commercial-model integrations in tools such as Cursor. Since the R1 wave, however, Cheah says Featherless’s coding share rose from under 5% in January to 20–30% by the period discussed. That change is why January’s distribution would already give the wrong impression of the platform.
For an open tooling stack, he pairs Cline for the chat-driven agent with Continue for autocomplete. In the February 2025 setup he describes, R1 supplies an experience he compares with commercial models in Cursor. This is an interaction comparison, not a demonstrated performance-equivalence test. Hobbyists can also pursue private, offline setups, including Mac Studio clusters. His practical recommendation is to try the projects in VS Code. The accompanying frame shows Cline’s completion report beside React code and a terminal displaying a successful compilation; it captures a visible end state, not a general reliability measurement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Graph workflows and alternative chat clients
Cheah assigns about 5% of traffic to ComfyUI and related personal agentic workflows. Graph interfaces familiar from diffusion and image generation also let people chain text operations. The users he names are not necessarily developers: reporters, lawyers, musicians, influencers and even poets assemble workflows to produce the material they need. As with coding agents, a handful of power users can generate substantial token volume. Unlike Cline’s sudden R1-related growth, this category has built gradually over months, amid competing platforms that remind him of JavaScript’s framework wars.
Cheah puts RAG and ChatGPT-style interfaces at about 20% of requests. Featherless has its own client, Phoenix, as other providers have theirs. His standout independent example is TypingMind: at talk time, a one-time purchase that users can run locally against an API provider such as Featherless. Its differentiators are product details—polished interaction, keeping pace with ChatGPT’s interface features and adding a plugin system. Running a client on a laptop does not by itself mean the model inference is local.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An email agent that can reach production
Cheah assigns the remaining agents-and-work category 10–20% of Featherless traffic, then divides it into narrow workflow automation and fully automated agents. For enterprise teams, the immediate objective is useful adoption with limited downside. Risk-averse institutions have little reason to accept a spectacular demonstration that cannot be controlled in production. His driver's-seat analogy expresses the requirement: build the human escape hatch from day one.
Insurance claims and logistics provide a concrete workflow because so much of the work arrives by email. The proposed system proceeds in a controlled sequence:
- Read an inbound email and gather the relevant inventory, rules and other business context.
- Draft a response using those checks.
- Present the draft in a UI where a person can edit, finalize or replace it.
- Let that person send the response.
The UI can sit on an existing Salesforce or ERP system. When the draft is right, the remaining task is simply to click send; when it is wrong, the person already has a place to take over.
The important implementation boundary is between creating a draft and authorizing an external action. A small TypeScript state model can express that boundary for an illustrative logistics email: a customer asks for 12 units, inventory shows 8, and a rule forbids promising more than the available stock. Preparing the response creates a reviewable proposal; it does not send it.
typescript
type InboundEmail = {
id: string;
requestedUnits: number;
};
type ReplyDraft = {
emailId: string;
body: string;
status: "needs-review";
};
type ApprovedReply = {
emailId: string;
body: string;
status: "approved";
reviewedBy: string;
};
function prepareDraft(
email: InboundEmail,
availableUnits: number
): ReplyDraft {
const offeredUnits = Math.min(email.requestedUnits, availableUnits);
return {
emailId: email.id,
body: `We can supply ${offeredUnits} of the ${email.requestedUnits} units requested.`,
status: "needs-review"
};
}
function approveReply(
draft: ReplyDraft,
editedBody: string,
reviewerId: string
): ApprovedReply {
if (!reviewerId.trim() || !editedBody.trim()) {
throw new Error("A reviewer and a response are required.");
}
return {
emailId: draft.emailId,
body: editedBody,
status: "approved",
reviewedBy: reviewerId
};
}
const email = { id: "email-42", requestedUnits: 12 };
const pendingReply = prepareDraft(email, 8);
// The review UI displays pendingReply; approval is a separate action.
The model-generated portion would supply the proposed wording; the surrounding application owns review and dispatch. In a deployed service, the server must enforce the approval boundary as well as the UI.
Cheah estimates that, when implemented well, this pattern can successfully draft around 80–90% of responses at launch, with humans taking over the remainder. Every submission still passes a person before reaching the customer. That is enough to improve productivity and make adoption attractive without requiring complete automation. Cheah places responsibility for missed corrections on the reviewer; operationally, the workflow must give that reviewer the context and ability to catch mistakes. A nominal approval button alone is not the same as effective review.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Earn automation one class of cases at a time
After the reviewed workflow has run across thousands of customers, teams can identify classes of cases that consistently work. They can then add classification and enable automatic replies for those cases. Automation becomes a permission earned by a known workflow, rather than a global setting applied before its behavior is understood.
The opposite approach can launch to enthusiasm and fail later. A fully automated workflow without an escape hatch eventually encounters an unusual case and sends a damaging response. Sometimes management has knowingly accepted that trade-off because the savings justify it. Otherwise, the incident can kill the project and postpone further adoption for another year. Cheah connects that risk to teams spending too long building complete automation before putting a limited, useful version into production. His objection to perfectly reliable autonomous agents concerns real production use, not whether a proof of concept can look convincing.
The practical target is the tractable majority of work, with escape hatches for the rest. Cheah uses 80% as that target and argues that, inside a large corporation, even partial automation can be worth millions in productivity and savings. Humans are not perfectly reliable either. The decision is whether the eventual mistake is acceptable and repairable, not whether the system can promise never to make one.
Two examples make the boundary explicit:
| Workflow | Failure the organization may accept |
|---|---|
| Cold outreach | Losing a lead it would not otherwise have contacted |
| Customer support | An error a human can later apologize for and correct |
These are choices about consequences. They do not imply that every lost lead or support error is harmless; they identify where a business might consciously accept imperfect automation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reliability needs a stable baseline
Cheah turns to reliability engineering, invoking Google SRE and safety-critical industries. Start with a streamlined system for the common cases. Then inspect the residual failures, solve a large fraction of those, and repeat. The mechanism is not simply retrying the same model ten times; it is repeatedly improving how the system handles the cases that remain unresolved.
He illustrates this with repeated 80–90% improvements to the remaining failure cases and cites 99.998% reliability after ten repetitions. That percentage is not derived in the talk. An idealized calculation makes the assumptions explicit: if each round fixes a fraction c of the remaining failures and never breaks a previously working case, then after n rounds:
Here F₀ is the initial failure fraction and Rₙ is the resulting success fraction. Real systems must measure both the remaining failures and regressions; the recurrence alone cannot establish production reliability. Cheah’s airline joke—repeat the process again—underscores the incremental approach. A less spectacular first launch can survive, deliver value and create the conditions for further improvement.
That brings the discussion back to old models. Stable dependencies let teams accumulate improvements, even if leaving models unchanged also gives their hosting provider continued business. Cheah nevertheless urges Featherless’s Llama 2 customers to evaluate an upgrade. He suggests a possible improvement from 99% to 99.9%, but does not name the metric or evaluation conditions. The useful distinction is between choosing an upgrade and having one imposed: Featherless’s promise to keep hosting the old model preserves the customer’s ability to decide.
His corresponding request to closed-model labs is Git-style versioning. A team cannot easily build successive reliability gains on a baseline that changes every week. Stable model versions and deliberate upgrades belong together: preserve the version that works, evaluate a replacement against the actual workload, and change it when the evidence supports doing so.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Qwerky and the next reliability problem
The closing announcement arrives through the Qwerky persona. It introduces Qwerky as a 72-billion-parameter hybrid of linear and attention transformers, claiming less than half the GPU compute cost of other transformer models and calling it the strongest post-transformer hybrid to date. The proposed alternative to attention-only models is motivated by inference cost and by the possibility that a different architecture can scale. Those comparative claims arrive without a specified workload or benchmark in the announcement.
The announcement puts Qwerky’s build cost at $100,000 and contrasts it with a stated $10 million for DeepSeek. The accounting scopes are not defined, so this is not a like-for-like comparison of total model-development costs. It is a claim about the feasibility of pursuing an alternative architecture with a smaller budget, not evidence that the two projects purchased equivalent training work.
The final question returns to usefulness. Being able to answer advanced mathematics or orbital-calculation questions is different from being trusted to complete one assigned task without hallucination or failure. The persona argues that MMLU becomes less informative for this purpose once models exceed the knowledge needed in ordinary office work. The gap is between answering difficult questions and behaving dependably inside a workflow.
The proposed research direction is to use linear-transformer models for persistent memories, customization and more reliable agents. These are possibilities to investigate, not capabilities established by the announcement. The concrete invitation is to discuss running Featherless or Qwerky in a private cloud or on premises—bringing the talk back to the user’s actual workload, the environment where it must run, and the behavior it must preserve.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Official weights, benchmark results, licensing and inference guidance for the original DeepSeek-R1 model.
Introduces the 12B Apache-licensed model, its NVIDIA collaboration, tokenizer and 128K context window.
Open-source coding agent with planning, clarifying questions and configurable approval controls.
Apache-licensed source code for the Continue coding assistant project.
Further reading
- How Hugging Face counts model downloadsDocumentation
Explains which file requests contribute to model download counters and why they differ from complete model transfers.
Updates since the talk
Later Featherless linear-attention model release with weights, inference instructions and comparisons against Qwen2.5-72B-Instruct.
- RADLADSPaper
Follow-up research coauthored by Eugene Cheah on converting pretrained transformers into linear-attention models at 7B, 32B and 72B scales.
Full transcript
Loading transcript…