AI Engineer Summit 2025

Frontier Feud

Read the talk

Frontier Feud: Guessing What AI Engineers Think

Who is the most influential AI researcher? In Frontier Feud, a plausible answer scores only if it matches a survey of 100 AI engineers. Barr Yaron tests the builders’ knowledge of their peers through trivia, model choices, and Fast Money.

From a talk by Barr Yaron, Mihir, John, Tina, Shrestha Basu Mallick, Paige Bailey, Colin Flaherty, Petra and Steven

Predicting what other AI engineers think

Amplify partner Barr Yaron opens Frontier Feud in New York with a challenge for the AI builders on stage: can they predict their peers’ answers? The game draws on a survey of 100 AI engineers. Following the Family Feud premise, contestants guess the responses, and more popular answers earn more points. A technically reasonable answer is useful only if it matches the board. Barr invites the audience to play along—and jokes that anyone unhappy with the answers should blame their neighbors.

First come introductions and hot takes. Mihir, from Anthropic, predicts that at least one of the roughly five major players training large models will stop by the end of that year. John, also from Anthropic, offers his opinion that AI systems make good therapists. Tina, from coding-agent startup Reflection AI, takes a different direction: everyone in the industry should spend about 20 minutes a day thinking with paper and pencil, without AI interruptions. That proposal earns applause.

Shresta, product lead for the Gemini developer APIs, builds on Tina’s suggestion. If models eventually represent people, she reasons, those people will need original thoughts worth training on. She saves her second hot take. Paige begins the other side’s introductions by clarifying that the contestants did not participate in the survey. Leading engineering for Google DeepMind’s AI developer relations team, she predicts that within a year and a half, most deployed models will run on devices: smaller, specialized models orchestrated together, rather than exclusive reliance on large models processing data elsewhere.

Colin, an Augment Code researcher previously working on AI for board games at FAIR, insists on delivering both of his predictions. Future dating apps will involve people’s AIs dating one another, and Transformers will eventually give way to models built with biological materials. Petra, who works on factuality for Google’s AI search answers, finds one-person, one-bot conversations boring; she imagines conversations with multiple bots instead. Steven, about to join Thinking Machines, closes by joking that anyone worried about hitting a token wall needs to improve their skills. The panel has plenty of opinions. Now it has to anticipate everyone else’s.

0:190:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:19 · section reference included

Influential researchers and an unforgiving board

Paige and Mihir approach the buzzer. Whoever presses first gets the first opportunity to answer: name the most influential AI researcher. Ilya Sutskever earns a match. Noam Shazeer does not, even as Barr acknowledges his influence. Two plausible names meet the same survey board, with different consequences.

Ilya? Ding. Noam? Not on the board.

Round one: 100 surveyed AI engineers were asked to name the most influential AI researcher. Here are the first two guesses.

FIRST GUESS
Ilya
Sutskever
On the board
NEXT GUESS
Noam
Shazeer
Not on the board
The board rewards the survey’s answers—not every plausible expert pick.

The successful side chooses to play rather than pass. John is invited to guess, and Andrej Karpathy earns another match. Geoffrey Hinton comes next—and initially gets a buzzer. The rejection prompts disbelief before Barr realizes she has misheard the name. She corrects herself and accepts the answer.

The next contestant can remember the Transformer paper, Attention Is All You Need, but cannot retrieve the author’s name. Naming the paper does not satisfy Barr’s request for a specific person. After reconciling the strike count with the earlier Hinton correction, she puts the team at one strike and moves on. Yann LeCun is the next successful guess.

The team is still learning the rhythm: it keeps answering because it keeps winning. Ashish Vaswani is subsequently offered as a specific name, but he is absent from the board. An ironic guess of Sam Altman also misses. With three strikes, the other team gets a chance to steal the accumulated points by supplying one matching answer.

The opposing side settles on Jeff Dean. He is absent too, leaving the round with Roko’s Basilisk. Before Barr can move on, contestants ask to see the remaining answers.

The reveal includes Andrew Ng, Fei-Fei Li, and Yoshua Bengio. Barr reminds the room that the board has only eight spots, while the list of influential researchers is much longer. Missing the board is a losing game answer, not a judgment on someone’s contribution to AI.

The survey board reveals the AI figures respondents named.Open full source frame
The survey board reveals the AI figures respondents named.
5:025:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:02 · section reference included

Cost comes first, and categories decide the steal

John and Colin take the next faceoff: what are the top considerations when choosing a model? Getting the buzzer to register becomes a contest of its own. Barr encourages more force, and the contestants give it another emphatic try. The first answer, intelligence, does not settle the faceoff; Barr says she will interpret the answers literally. Safety follows and is explicitly absent from the board.

Price breaks the deadlock. Cost is the number-one survey answer for considerations when choosing a model. Barr admits that this surprises her. Latency then earns a match.

Cost and pricing appear among the model-selection survey answers.Open full source frame
Cost and pricing appear among the model-selection survey answers.

Evaluation and benchmark scores require a judgment call: she accepts them under accuracy/performance, while acknowledging that the categories are ambiguous. The accepted answers now cover expense, response time, and performance.

A guess about the provider’s CEO gets a laugh and a buzzer. So does an answer about where the model is served, including whether it runs on premises. The opposing team gets its opportunity and chooses open source versus closed source. That answer succeeds. The successful steal earns 63 game points. The distinction matters to the result: serving location was not accepted, while openness was treated as a matching category. The award is the round’s stolen points, not a respondent count for that one answer.

Barr announces a close game heading into the final main question, but the contestants insist on seeing the answers first. During the reveal, capabilities is mentioned. The room keeps the host from advancing too quickly; everyone wants to know what else they could have guessed.

9:299:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:29 · section reference included

Agents make the board; AGI takes the top spot

Petra and Tina come forward, now thoroughly briefed on the force the buzzer requires. This time it sounds before Barr finishes the question: name a buzzword everyone in AI is tired of hearing. Agents earns a match, but it is not the top answer. The other contestant still has a chance to take control with a more popular response. The countdown expires without another guess.

Barr turns to Shresta, whose answer is multimodality. It misses, and Barr corrects the displayed strike count. Copilots follows, attracting encouragement before another rejection. Then comes AGI. Barr briefly pretends it has missed too before revealing the match. AGI is the number-one answer to the buzzword question.

DeepSeek is the next guess and does not score. Barr jokes that it might fit for that particular week. With a steal available, the opposing side returns to safety—an explicit callback to the previous round. It is absent here as well.

The reveal adds RAG and prompt engineering. The question measures fatigue with terminology, not whether those techniques are useful. With the steal unsuccessful, Barr declares Roko’s Basilisk the winner of the main game and asks for two representatives for Fast Money.

The buzzword round reveals AGI, agents, RAG, and hallucination.Open full source frame
AGI, agents, RAG and hallucination appear on the buzzword board.
13:2913:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:29 · section reference included

Four answers put Mihir on 140 points

Mihir and John represent the winning team. John leaves the stage while Mihir goes first. Their combined Fast Money target is 200 points. Mihir is given 20 seconds to answer five questions, with the option to pass and return if time remains. Barr tests the buzzer before beginning.

The first prompt asks for an AI tool engineers love. Mihir answers Cursor. Then Barr realizes she has not started the clock. Amid laughter, she starts the countdown and moves to the next question.

Mihir supplies three more answers in quick succession:

  • Job most at risk of AI disruption: software engineers.
  • Most influential AI paper in history: Attention Is All You Need.
  • An AI engineer’s biggest nightmare at 2 a.m.: hardware failure.

The buzzer sounds after the last response.

Mihir’s pass reaches four questions. The scoring reveal follows their original order. Cursor is the number-one answer for an AI tool engineers love. Software engineering is the number-three answer for the job most at risk of AI disruption. Attention Is All You Need is the number-one answer for the most influential AI paper in history, with Barr emphasizing its substantial lead. Hardware failure receives credit under infrastructure problems: another specific answer accepted within a broader category.

Barr announces a first-pass total of 140 points and calls for John.

16:4716:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:47 · section reference included

Repeated answers consume the second chance

John returns after Barr jokes that he has run away. John is given 25 seconds to answer five questions. This time a buzzer can mean either that the answer is absent from the board or that Mihir has already used it. Passing remains an option. Barr starts the timer.

The duplicate rule immediately bites. John also chooses Cursor, hears the buzzer, and switches to ChatGPT. For the job at risk, he answers artist. Asked for the paper, he arrives at Attention Is All You Need—another answer already taken.

During the scoring reveal, Barr announces model APIs as the board’s third answer for the tool question. She then accepts artist under content creation/writing, acknowledging the ambiguity. The repeated paper answer cannot add to Mihir’s result.

Barr lets John attempt the 2 a.m. nightmare question after the timed sequence. Before he answers, she shares a survey write-in about receiving a cold email from a venture capitalist—and promises a conversation with whoever submitted it. John offers CUDA errors.

Barr jokes that the teammates think alike, then reveals the leading responses across all five questions:

  • Cursor is the top answer for an AI tool engineers love.
  • Data entry is the top answer for the job most at risk of AI disruption.
  • Attention Is All You Need is the top answer for the most influential AI paper.
  • An outage, of the model or otherwise, is the top answer for an engineer’s 2 a.m. nightmare.
  • Healthcare is the top answer for the industry that would benefit most from AI.

These are the survey’s answers to the game’s prompts, including its questions about future disruption and benefit.

The fifth question, about an industry that would benefit from AI, arrives only in this final reveal. Then comes the losing sound: the team does not reach the 200-point Fast Money target.

18:5718:59
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:57 · section reference included

From llama prizes to tools and workflows

Missing the bonus target does not stop the prize ceremony. Barr announces a massive llama for one team, rainbow llama Beanie Babies for everyone, AI engineering books, and gift cards for New York restaurants. There is plenty to take home even without the Fast Money victory.

The closing invitation moves beyond the game’s teaser questions. Barr announces a larger survey on the state of AI engineering, with findings to be presented at the upcoming June AI Engineer World’s Fair. Its subjects will include the tools engineers use and the workflows they follow. She frames participation as a way to make the industry more transparent and directs the audience to the event’s survey link or QR code.

21:0721:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

21:07 · section reference included

Resources

From the talk

  • Barr Yaron's separate survey report adds context about engineering practices beyond the game's popularity questions.

Transcript download
Full transcript

Loading transcript…