AI Engineer World's Fair 2025

The Robots Are Coming for Your Job, and That's Okay

Elmer Thomas· Principal Developer Educator, TwilioMaria Bermudez8:57

Loading slides…

Read the talk

The Robots Are Coming for Your Job, and That's Okay

Twilio’s documentation workflow uses focused AI helpers to propose edits, generate metadata and simplify prose, while validation and human review keep publication under team control.

From a talk by Elmer Thomas and Maria Bermudez

Before you start: Familiarity with Markdown, pull requests and automated checks will help you follow the documentation workflow.

A small docs team facing a flood of requests

A small documentation team sits downstream of a much larger organization: Jira tickets keep arriving, first drafts need corrections, and routine grooming competes with work that requires editorial judgment. The goal for Twilio’s team is to use AI to absorb some of that repetitive work without replacing the people responsible for clarity and accuracy.

Elmer Thomas, Principal Developer Educator at Twilio, introduces the workflow alongside Maria Bermudez, a lead developer of the team’s AI Docs Buddy. Thomas describes error-prone first drafts arriving from more than 100 product teams. Style checks, image alt text and SEO metadata consume additional time. Generative AI offers help with those chores, but unconstrained generation introduces another problem: plausible content that may be wrong.

Slide titled “The Pain” shows a docs team behind a help desk surrounded by people holding Jira papers.Open full source frame
The docs team faces a crowd of Jira requests.
0:110:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:11 · section reference included

Give each helper a narrow job

The team organizes the work into six single-purpose helpers behind a simple Next.js front end. Each has a bounded output that a person can inspect.

HelperAssigned job
Automated editorGrammar, formatting and accuracy edits
Image alt text generatorAccessibility descriptions
Jargon simplifierDeveloper language into plain English
SEO metadata generatorTitles and descriptions within character limits
Docs outline builderNavigation and structure recommendations; coming soon
Slack botHelp-channel request triage

The outline builder is planned work at the time of the recording. The broader division of labor is already clear: repetitive jobs go to specialized helpers, leaving humans to concentrate on judgment and clarity.

1:251:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:25 · section reference included

Turn generated changes into reviewable proposals

Repeatable, high-volume work with little need for creativity is the starting point. A request enters through the Next.js UI and reaches a custom agent, with the model chosen for the particular job. The agent uses the team’s style guide and rubric, retrieved from Airtable so collaborators can maintain the guidance together. Editorial policy therefore has an explicit home outside the individual request.

The proposed changes then pass through a validation layer containing Vale linting and CI/CD tests. Vale checks configurable prose rules; it does not establish whether a product claim is true. A GitHub pull request makes the changes available for scrutiny through code-owner review. GitHub treats requesting that review and requiring approval before merge as separate settings.

Product and engineering reviewers usually participate as well, and a human merges only after accepting the changes. Generating an edit does not authorize publication. Thomas initially describes the layers as crushing hallucinations, then corrects himself: the claim is that they significantly reduce them. The protection comes from combining automated checks with several people’s judgment, rather than asking any single layer to guarantee correctness.

“Under the Hood” diagram links Next.js UI to a custom GPT-4/o1 agent, validation layer, GitHub PR, and human in the loop.Open full source frame
The request flow connects the Next.js UI, agent, validation, GitHub PR, and human review.
2:162:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:16 · section reference included

Inspect both the edit and its reason

Bermudez begins in the application’s overview and release notes page, which the team uses Copilot to generate. From there, she opens the automated editor. It accepts an MDX file, a Markdown file or a live URL; she chooses the URL option. This lets the editing pass start with an existing published page rather than requiring someone to prepare a separate text input.

The editor uses o1, and the interface displays a loading graphic while it works. Bermudez reports that experimentation identified it as the best model for consistently applying this team’s style guide and rubric. That choice carries a visible latency cost in the interaction.

The result offers two complementary ways to review the proposal. A diff shows what changed. The Changes Made tab provides the original text, revised text and an explanation connecting the change to a style-guide instruction or rubric item. The reviewer can inspect both the textual effect and the rule offered as its justification before deciding whether to accept it.

3:514:02
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:51 · section reference included

Run a separate metadata pass

The editor’s output also exposes a limit: it sometimes catches a missing SEO description and sometimes does not. Bermudez moves the same page into the dedicated SEO metadata generator. Specialization gives the missing field its own explicit task instead of relying on the general editing pass to notice every omission.

She uploads a file and returns to the Conversational Intelligence page that was just edited. The generator produces meta titles by default—users must specify if they do not want them—and a meta description that accounts for character limits. The handoff is performed by Bermudez through the interface, keeping the metadata operation separate and visible.

4:535:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:53 · section reference included

Generate alt text across multiple pages

The alt text generator also supports a live URL, but Bermudez demonstrates selecting multiple pages and generating descriptions for all their images in one action. Batch input reduces the repeated work of submitting each image separately.

The results display image filenames and previews beside generated text. Bermudez calls attention to the quick completion and to a second practical requirement: the output conforms to the documentation platform’s expected format. Producing a description is only part of the task; producing one in the form the publishing workflow accepts makes the result easier to use.

Image Alt Text Generator results show image filenames, previews, and generated text in adjacent panels.Open full source frame
Generated alt text appears beside uploaded image previews.
5:525:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:52 · section reference included

Make revisions easy to reuse in a pull request

The jargon simplifier serves both authors drafting a page and reviewers working through a pull request. Bermudez pastes prepared text and selects Simplify. The result includes a diff, like the automated editor, plus a separate revised-text tab. A reviewer can copy that text into a pull request comment or use it to edit the file directly.

Jargon Simplifier displays original text above the selected “Revised Text & Explanation” tab, with revised text, a copy icon, and a collapsed “Explained Jargon” section.Open full source frame
The jargon simplifier presents revised text with a copy control.

For example, a Markdown review suggestion can make a proposed simplification directly usable:

markdown

Replace:
Utilize the configuration interface to modify notification preferences.

With:
Use settings to change your notification preferences.

The useful handoff is a concrete revision that a reviewer can accept, adjust or reject. It remains a proposal until someone applies it.

These helpers are not yet an autonomous chain. Bermudez closes the walkthrough by saying the team wants the agents to communicate with one another, but substantial work remains. The demonstrated workflow consists of focused tools with a person moving between them.

6:256:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:25 · section reference included

Maintain quality through distinct feedback loops

Tools alone do not guarantee quality. Thomas separates the remaining risks by the kind of review they need:

  • Hallucinations: Combine Vale linting and CI tests with scrutiny from human stakeholders.
  • Bias: Use dataset tests and prompt audits.
  • Stakeholder misalignment: Review pull requests weekly, accelerating the cycle to days or hours when needed.

These checks cover different failure modes; passing a style check does not settle a factual question or establish that stakeholders agree with the result.

Slack feedback loops with product managers and engineering provide another route for corrections. The team uses those exchanges to tune prompts continuously. Quality maintenance is therefore an ongoing collaboration around actual outputs, rather than a one-time prompt-writing exercise.

7:207:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:20 · section reference included

Start with one throughput bottleneck

The adoption process begins smaller than the full set of helpers:

  1. Identify one pain point that is limiting throughput.
  2. Choose a single task that is repeatable and rule-based.
  3. Loop with its users at least weekly.

Ship, measure and refine that workflow. Thomas expects several such improvements to accumulate into greater team velocity; the recording offers that as the direction of travel, without a measured team-wide gain. The practical starting point is one bounded job whose outputs people can review and whose shortcomings can feed the next iteration.

8:128:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:12 · section reference included

Resources

From the talk

Updates since the talk

  • Reusable prompts for documentation editing, jargon simplification, alt text and SEO metadata, with examples and integration guidance.

  • Twilio's announcement explaining how to adopt and contribute to its public documentation prompt templates.

  • Making Alt Text FastArticle

    Maria Bermudez explains Twilio's batch alt-text workflow, human review and subsequently reported production results.

Transcript download
Full transcript

Loading transcript…