AI Engineer Europe 202616:08
Benchmarking semantic code retrieval on Claude Code
Kuba Rogut
Infrastructure & deployment · Coding agents · Retrieval-augmented generation
← All topics
AI Engineer topic
Benchmarks
24 AI Engineer conference talks about Benchmarks.
AI Engineer World's Fair 202515:44
Benchmarks Are Memes: How What We Measure Shapes AI—and Us
Alex Duffy
Evals · Benchmarks · Post-training
AI Engineer World's Fair 202612:49
Benchmarks: The Good, the Bad, and the Ugly
Ali Khial
Evals · Retrieval-augmented generation · Coding agents
AI Engineer World's Fair 202620:30
Beyond Static Intelligence: Evaluating Continual Learning
Parth Asawa
Retrieval-augmented generation · Evals · Safety & governance
AI Engineer World's Fair 202617:34
DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve
James Shi
Evals · Developer workflows & testing · Coding agents
AI Engineer World's Fair 202518:28
Measuring AGI: Interactive Reasoning Benchmarks
Greg Kamradt
Evals · Enterprise AI · Agent engineering
AI Engineer World's Fair 202623:35
Stop Evaluating Models Like It's the 50s - Alejandro Vidal, Mindmakers
Alejandro Vidal
Observability & reliability · Evals · ML fundamentals
AI Engineer World's Fair 202612:58
SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI
Rishi Desai
Evals · Coding agents · Benchmarks
AI Engineer Europe 202616:30
SWE-rebench: Lessons from Evaluating Coding Agents on Real Software Engineering Tasks — Ibragim Badertdinov, Nebius
Ibragim Badertdinov
Agent engineering · Coding agents · Benchmarks
AI Engineer Europe 202623:25
The Art & Science of Benchmarking Agents
Vincent Chen
Evals · Safety & governance · AI engineering leadership
AI Engineer World's Fair 202511:20
The Benchmarks Game: Why It's Rigged and How You Can (Really) Win
Darius Emrani
Evals · Safety & governance · Infrastructure & deployment
AI Engineer World's Fair 202514:10
Vector Search Benchmark[eting]
Philipp Krenn
Evals · Retrieval-augmented generation · AI cost & performance
AI Engineer Europe 202620:24
What Do Models Still Suck At?
Peter Gostev
Evals · Retrieval-augmented generation · ML fundamentals
AI Engineer World's Fair 202617:25
When Will The Benchmaxxing Plague End?
Nick Heiner
Infrastructure & deployment · Evals · Retrieval-augmented generation
AI Engineer World's Fair 202618:28
The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev
Jo Kristian Bergum
Search & retrieval · Hybrid search · Benchmarks
18:42
One Day, $1,000 in Tokens: An AI War Room for Legacy Code — Ford Prior, GEICO
Ford Prior
Coding agents · Agent orchestration · Enterprise AI
AI Engineer World's Fair 202616:07
Are LLM Performance Benchmarks Reliable? — Ashok Chandrasekar & Jason Kramberger, Google
Ashok Chandrasekar · Jason Kramberger
Benchmarks
AI Engineer World's Fair 202652:30
From Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases
Wolfram Ravenwolf
Evals · Coding agents · Observability & reliability
AI Engineer World's Fair 202619:38
We Let Claude Code and Codex Race Human Researchers — Elie Bakouch, Prime Intellect
Elie Bakouch
Autoresearch · Coding agents · Benchmarks
17:42
Same GPU, 50% More Output: Why the Operator Matters — Oliver Kennon, SemiAnalysis
Oliver Kennon
AI cost & performance · Inference · Infrastructure & deployment
AI Engineer World's Fair 202630:00
Can LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AI
Simran Arora
GPU programming & kernels · Benchmarks · Coding agents
AI Engineer World's Fair 202621:03
Building the Document Context Layer for AI Agents — Jerry Liu, LlamaIndex
Jerry Liu
Document understanding & OCR · Retrieval-augmented generation · Benchmarks
26:41
Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa
Ryan Ahern
Search & retrieval · Evals · Agent engineering
22:25
How long can your skills be before your agent forgets what you told it? — Laurie Voss, Arize AI
Laurie Voss
Benchmarks · Context engineering · Evals