AI Engineer Summit 2025

Your AI Agent Isn't an Engineer: The Art of Thoughtful Anthropomorphism

Rizel Scarlett· Staff Developer Advocate, Block20:20

Loading slides…

Read the talk

Your AI Agent Isn’t an Engineer: Thoughtful Anthropomorphism in Practice

Calling an agent an engineer sets expectations its tool calls cannot fulfill. Better developer marketing starts with understanding the system, demonstrating useful work, and preserving developer control.

From a talk by Rizel Scarlett

Before you start: No machine-learning background is required; basic familiarity with developer tools will help with the agent-loop example.

Will AI replace software engineers?

Will AI replace software engineers? The question can provoke stubborn resistance—sticking with COBOL and refusing to learn anything about AI—or a scramble to take every course and apply for every AI job. For Rizel Scarlett, speaking as a staff developer advocate at Block after previously working on GitHub Copilot, this is a practical problem for developer relations: how do you invite developers to use a tool while its marketing threatens their livelihood?

A slide headed “Raise your hand…” asks whether viewers have been personally victimized by the question “Will AI replace software engineers?” beside a crowd raising their hands.Open full source frame
Will AI replace software engineers?

The anxiety has consequences beyond social media. Scarlett cites Salesforce’s hiring plans as an example. In December 2024, reporting on Marc Benioff’s remarks attributed a greater-than-30% engineering productivity improvement to Agentforce and other AI tools. Benioff connected that claim to plans for no additional software engineers and fewer support engineers in 2025. The report supplies neither a defined productivity measurement nor independent validation; the staffing figures describe plans, not established outcomes.

Scarlett’s usual advice is to upskill and adapt: nobody wants to keep driving a horse and buggy after everyone else has moved to cars. She also finds AI useful for understanding unfamiliar technologies and quickly prototyping ideas. But those benefits leave a different question unanswered: why does selling a helpful tool so often require presenting it as a replacement for a person? Building developer trust without compromising integrity begins there.

0:000:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

From an additional tool to a substitute employee

Scarlett locates part of the replacement narrative inside the industry itself: developers, marketers, and developer relations teams helped construct it. Her hypothetical sales pitch starts with an executive already paying engineers hundreds of thousands of dollars—$300,000, then escalating toward $400,000 and $500,000. Asking that executive to buy another tool sounds like adding an expense. Promising to eliminate some engineering costs makes the same purchase sound like a saving.

That shift rewards a quick sale, but it also changes the product’s implied contract. The customer is no longer buying software that helps an engineer; they are buying something presented as capable of taking the engineer’s place. A promise of reduced costs and increased productivity becomes an invitation to describe the tool as human.

2:322:48
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:32 · section reference included

Familiarity helps until it promises too much

Anthropomorphism means assigning human traits to nonhuman entities. It is not inherently harmful: familiarity can make a digital experience easier to approach. Scarlett connects this to other familiar interface cues. An e-book can animate a turning page and play a paper sound even though there is no paper. In a Bluesky exchange she recalls, Sunil Pai offered electric cars’ prerecorded engine sounds as another example: a recognizable cue helps a driver feel connected to an unfamiliar mechanism.

An “Electric Cars” slide lists “pre-recorded engine sounds” beside an illustration of a green car at a charging station under solar panels.Open full source frame
Electric cars and pre-recorded engine sounds.

AI makes that translation especially tempting. Its academic history and opaque operation leave many users without a clear picture of either its capabilities or its limits. Human descriptions provide a shortcut: Scarlett points to an airport advertisement presenting Claude as a friend, Devin’s introduction as an AI software engineer, and ChatGPT status messages describing thinking or reasoning instead of loading.

The problem is the expectation attached to the shortcut. A familiar cue can help someone operate a tool; a human job title can imply that it thinks, judges, and works independently like a person. Scarlett identifies three consequences: alienating developers, creating unrealistic expectations, and obscuring the actual value of agents.

3:353:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:35 · section reference included

The replacement pitch undermines adoption

Calling AI an engineer encourages decision makers to treat it as a one-to-one substitute for human talent. Yet developers are also the people most likely to become skilled users and advocates. In the 2024 Stack Overflow Developer Survey, 76% of respondents reported using or planning to use AI tools in development. That combines adoption with intent; it does not mean 76% were already using them. Asking those developers to improve a product through use while telling them it will eventually take their jobs creates an obvious conflict.

Scarlett turns to Wired’s “Game Developers Are Getting Fed Up With Their Bosses’ AI Initiatives” for evidence of that tension. In the GDC survey it covered, 52% of respondents worked at companies implementing generative AI, while 30% judged its impact on the industry negatively. The first figure describes respondents’ workplaces, not the share of all companies or the presence of AI content in shipped games. One anonymous respondent with a PhD in AI regretted contributing to the algorithms; another wanted generative AI to help people work faster instead of costing them their jobs.

The same framing creates a performance problem. Describe a product as a mid-level software engineer and users expect human-level work. Scarlett instead describes a non-sentient system trained on historical patterns that can miss context, hallucinate, and produce different outputs across attempts. When those limitations collide with promises of perfection or superiority, disappointment damages both product and company credibility. Developers want an accurate account of what they are adopting.

The useful promise is parallel work. Automating tedious tasks, accelerating prototypes, and helping with debugging can leave developers more time for creative problem-solving and higher-impact contributions. As agents gained attention during the 2024–2025 period, their ability to execute shell commands, create calendar events, or build applications made butler and assistant metaphors increasingly tempting. Their ability to act makes precise explanations more necessary, not less.

6:056:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:05 · section reference included

Become customer zero, then explain the loop

Thoughtful anthropomorphism starts with understanding the product. Scarlett’s framework connects that understanding to naming, augmentation, transparency, developer control, demonstrations, documentation, and open collaboration. These are not responsibilities reserved for the engineers who built the system. Developer relations, social media, communications, executives, and sales all need enough firsthand knowledge to explain what they are selling.

Become customer zero: use the product extensively before release, then keep experimenting as features change. Learn what LLMs can do, how copilots differ from agents, and how tokens, context management, and tool calling affect operation. Those details make it possible to troubleshoot an error instead of repeating a slogan. The agentic loop is a useful starting point, although different agents use different versions of it.

Consider Scarlett’s example request: generate tests for a web application. The agent passes that request, together with its available tools, to an LLM. She names Claude Sonnet 3.5 and “GPT-4.0” as possible models; the latter is the model label used in the recording. The model proposes a plan involving a test-generation tool, perhaps using Cypress or Playwright: create a folder, create files, import dependencies, and construct a test suite.

For a concrete Playwright example, suppose the requested suite checks that a tic-tac-toe page starts with nine empty buttons inside #board. A proposed test file could be:

javascript

import { test, expect } from '@playwright/test';

test('starts with an empty tic-tac-toe board', async ({ page }) => {
  await page.goto('/tic-tac-toe');

  const cells = page.locator('#board button');
  await expect(cells).toHaveCount(9);
  await expect(cells).toHaveText(Array(9).fill(''));
});

This gives the plan a specific artifact to create and inspect. Producing the file is only one step in the loop.

The division of work is:

  1. Plan: The LLM proposes actions using the tools available to the agent.
  2. Execute: The agent carries out the plan and its tool calls.
  3. Check: The agent sends the resulting information back to the LLM for assessment.
  4. Revise or deliver: If the result needs work, the plan changes and the loop continues. Otherwise, the agent returns the result to the user and awaits another request.

In the test example, a proposed file, a created file, and a checked result are distinct states. Describing them separately makes the system easier to understand than saying an engineer has handled the task.

9:409:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:40 · section reference included

Goose makes a move—and loses

Scarlett makes the loop visible by playing tic-tac-toe with Codename Goose, the agent made by her company, Block. After she announces that she has played, Goose passes that information to the LLM. The model directs it to capture the screen, inspect the board, and decide where and how to move. Goose uses its screen-capture tool and then HTML to make its move.

There is a noticeable wait while the agent communicates with the model. Eventually it places an O. Scarlett wins the game; the displayed board ends with three X marks down the middle column and an X-wins message. The demonstration shows an agent taking action through tools without turning that action into a claim of flawless judgment.

A tic-tac-toe board displays “X wins!” with three X marks down the middle column, beside the Goose conversation and screen-capture output.Open full source frame
The tic-tac-toe demo ends with “X wins!”

The session also exposes a more useful failure than simply losing a game. At points, Goose could not obtain a screen capture. It had to retry and revise its plan. That recovery is part of how the system works: the model’s next action depends on information returned by tools, and acquiring that information can itself fail.

12:4012:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:40 · section reference included

Choose names that leave the developer in control

Use human framing sparingly. Scarlett prefers nonhuman names and terms such as copilot, agent, or assistant over an AI-engineer title. The objection is not that humans never make mistakes—she explicitly acknowledges that they do—but that the title encourages expectations the product does not reliably satisfy.

GitHub Copilot is her positive naming example: a copilot helps someone do the work while leaving that person in control. At the time of the talk, she describes the product as moving from its earlier assistance model into an agentic mode. The naming lesson survives that distinction: communicate the relationship between user and tool, not just an impressive occupational identity.

That relationship matters because developers rarely have only one responsibility. In “The Average Developer is a Multitasker: A Case for Agents”, Scarlett develops the same augmentation argument around people who are also parents, maintainers, and instructors. Parallel work is useful because it complements those responsibilities. The agent belongs in a developer’s toolkit; its value does not require claiming it can replace the person using it.

13:3713:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:37 · section reference included

Explain the system before others explain it for you

Transparency gives users a better account of the mechanism before speculation fills the gap. Scarlett recommends open source where possible, or white papers and conference explanations when it is not. During GitHub Copilot’s transition from beta to public availability, she listened to Twitter Spaces in which people tried to reverse-engineer its behavior through discussion. She was not onstage and was unsure whether she could share implementation details that were not yet public.

One theory claimed that, as a developer typed, Copilot searched repositories, stole code, and returned it. Scarlett rejects that as an explanation of how the product worked. Her operational lesson is to make an explanation available early enough that speculation does not become the default account.

Developer control makes that transparency practical:

  • Model choice: Let users select the LLM.
  • Behavior: Let them adjust responses, including verbosity or concision.
  • Inspection: Expose debugging logs so they can investigate problems.
  • Integration: Connect to APIs and the workflows developers already use.

Scarlett points to Goose as an example: it is open source and uses Model Context Protocol, or MCP, servers as extensions. She names Figma and JetBrains as examples of the tools those connections can bring into a workflow. The point is to let developers shape and inspect the system instead of asking them to accept a fixed personality.

15:0115:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:01 · section reference included

Show a specific task, including the difficult part

An agent that can supposedly do everything leaves a prospective user with two unanswered questions: what can it actually do, and how do I make that happen? Short videos, GIFs, and blog posts can answer both through bounded tasks: creating a test suite, converting code between languages, turning wireframes into interfaces, or turning comments into documentation.

Live demos add another kind of instruction. When a generation fails, the audience can watch the presenter diagnose and recover from it. A polished demonstration still has a place, but seeing someone work through a familiar difficulty can teach more than a seamless result. The failure becomes useful when the demonstration shows how to proceed.

A slide lists short videos, GIFs, blogs, test suite creation, code conversion, UI transformation, documentation generation, and live demos as teaching moments, beside a tweet and replies.Open full source frame
“Show, Don’t Tell”: specific examples and live demos.

Scarlett’s recurring GitHub Copilot demo connected to Twitter’s API and posted a tweet saying she had written it with Copilot and was at the current conference. This was a historical API demonstration, not a current integration recipe. Its strength was the visible, shared outcome: audience members encouraged her when Copilot generated the wrong thing and celebrated when the post finally worked.

The result also reached people outside the room. They saw the tweet and wanted to know how she had made it with Copilot. A React Miami organizer even framed one. A small task had become memorable because people understood the goal, witnessed the difficulty, and recognized the successful result.

16:4716:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:47 · section reference included

Help users repeat what works

A memorable demonstration needs a path to independent use. Installation guides get people started; prompt playbooks give them working approaches to try; explanations of data use help them make informed decisions. Documentation turns initial excitement into the ability to operate the product.

Community spaces extend that learning. In GitHub Discussions or Discord, users can share a prompt, compare results, and help each other troubleshoot. People who discover useful patterns can become advocates because they have something concrete to teach.

Scarlett’s example is cursor.directory, a community-created collection rather than a company project. In the period described in the talk, it collected prompts for Cursor rules that other users could copy and adapt. She places it alongside her enthusiasm for Cursor’s IDE and composer agent: sometimes the most useful starting point is simply seeing what worked for someone else.

The practices reinforce one another. Understanding the mechanism improves the explanation; honest names set expectations; control and transparency let developers investigate; demonstrations, documentation, and community examples help them succeed. How AI is presented shapes how people understand it. Moving beyond replacement messaging means giving developers a credible account of what an agent can do—and a practical way to make it useful.

18:2618:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:26 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…