▶ Watch ↗AI Engineer World's Fair 202518:28
Conference affiliation: ARC Prize Foundation · 2025
Greg Kamradt is president and a board member of the ARC Prize Foundation, where he builds benchmarks and open competitions to test whether artificial intelligence can learn unfamiliar skills. He also created Needle In A Haystack, an open-source evaluation exposing the difference between a language model’s advertised context window and its ability to retrieve information within it.
Kamradt worked in finance at NetApp, then in strategy, growth, and predictive analytics at Salesforce before leading operations and product at Digits, an AI-focused accounting company. He founded Leverage, an AI product and education studio, and taught developers to build language-model applications through tutorials, open-source material, and practical demonstrations.
His Needle In A Haystack evaluation places specific facts at varying depths inside increasingly long inputs, then measures whether models can recover them. The project now includes multiple-fact retrieval and chained reasoning, making long-context performance testable beyond headline token limits.
In early 2024, Zapier co-founder Mike Knoop recruited Kamradt to help organize ARC Prize, an open competition based on François Chollet’s Abstraction and Reasoning Corpus. Working with Knoop, Chollet, and Bryan Landers, Kamradt shifted from building AI applications toward evaluating whether models genuinely acquire new capabilities. He subsequently co-authored the 2024 and 2025 ARC Prize technical reports.
In March 2026, Kamradt launched ARC-AGI-3, the foundation’s first fully interactive benchmark, alongside competitions offering more than $2 million in prizes. His subsequent analysis of frontier-model reasoning traces documented how capable systems can recognize isolated effects without understanding an environment. Among early competition winners, he highlighted agents that wrote and executed Python to investigate unfamiliar games instead of depending on elaborate predefined scaffolding.
Kamradt also continues building practical AI products, including agnts.sh and ReadSide: tools for serving agent-readable web content and discussing specific passages while reading.
▶ Watch ↗AI Engineer World's Fair 202518:28
Loading more talks and highlights…