← All speakers

George Cameron

Conference affiliation: Co-Founder · Artificial Analysis · 2026

George Cameron is co-founder and chief product officer of Artificial Analysis, the independent AI benchmarking company he founded with Micah Hill-Smith. His work helps developers compare model capability with the cost and time required to complete useful work. That means evaluating both the model and how it is served: inference providers, hardware and caching can change the economics of the same application.

On this page

From infrastructure strategy to model evaluation

Before founding Artificial Analysis, George was a senior strategy consultant at Altman Solon, specializing in technology companies and data centers. Artificial Analysis began as a side project in 2023 and launched publicly in early 2024. In their account of the company’s origins, Micah described a model-selection problem he encountered while building a legal AI research assistant: different stages needed different levels of capability, balanced against speed and cost. Together, the founders built comparisons across models and providers to help developers make those decisions.

The company expanded into benchmarking agents, language models, inference infrastructure, accelerators and modalities including image, video and speech. Its Intelligence Index provides a general comparison of model capability; its State of AI research examines model economics, hardware and enterprise adoption.

George’s product work also includes Optima, which brings evaluation to a team’s own workload. Teams can supply examples, import agent traces or provide context from a coding environment to build custom benchmarks, then compare the quality of completed work with cost and time per task. A general leaderboard can narrow the options; an application-specific benchmark tests whether those options meet the team’s actual requirements.

Knowledge, guessing and useful capability

George co-authored AA-Omniscience with Declan Jackson, William Keating and Micah Hill-Smith. Their November 2025 paper evaluated factual knowledge and the willingness to acknowledge uncertainty across 6,000 questions. Correct answers earned credit, incorrect answers incurred a penalty and abstentions were neutral. Only three of the 36 models tested achieved a positive score at launch.

The scoring makes a practical distinction: a model can answer more questions correctly while also making more unsupported guesses. The team separated factual knowledge from knowledge reliability and found that general intelligence rankings did not reliably predict factual knowledge or hallucination behavior. Their work gives developers another reason to examine the capability that matters to an application rather than treating an overall ranking as a complete purchasing decision.

The economics of completed work

George’s explanations of model selection follow the costs that accumulate beyond a single advertised token price. The 2025 Trends Across the AI Frontier recording, credited to George and Micah, compared output consumption on the same Intelligence Index evaluations: approximately 7 million tokens for GPT-4.1, 72 million for high-reasoning o4-mini and 130 million for Gemini 2.5 Pro. The price per output token therefore tells only part of the story; the number of tokens a model generates also determines the bill.

In his 2026 analysis of agent costs, George extended that reasoning to long workflows. On AA-Briefcase, where agents complete knowledge-work tasks in an environment containing thousands of files, Opus averaged roughly 165 steps at about 13 cents per call, or $21 per task. He estimated that the same workload would exceed $200 with no cache hits. His advice centers on three quantities: input tokens per call, the proportion served from cache and calls per task. Reasoning effort can affect all three: it changes the reasoning generated, can change the number of turns, and adds material that subsequent calls receive in their message history.

Two further distinctions shape his recommendations:

  • Sequential latency changes what developers can build. The 2025 frontier analysis illustrated this with 30 consecutive requests: at ten seconds each, they take five minutes; at one second each, they take 30 seconds. Faster calls can make an interactive workflow viable or leave room for an agent to take more steps within the same waiting time.
  • Some tasks have a capability threshold; others keep benefiting from stronger reasoning. George contrasts retrieving last month’s subscription total with analyzing how to reduce a business’s costs. A cheaper model with the appropriate tools may already retrieve the correct total. The analysis may improve with a stronger model. The first decision is how much capability the task needs; the next is what additional improvement is worth paying for.

George also tracks how efficiency expands the available choices. The 2025 frontier analysis examined sparse architectures, distillation, inference software and specialized accelerators alongside the narrowing gap between open-weight and proprietary models. In his 2026 talk, he estimated that open-weight models were reaching capability levels attained by leading proprietary models three to nine months earlier. He nevertheless expects compute demand to grow: stronger reasoning and agents making many sequential requests can absorb improvements in inference efficiency. For developers, those gains create opportunities to attempt more demanding work while continually reassessing which model and provider can deliver it economically.

Read the topics behind these talks

2 conference talks

Loading more talks and highlights…

References