AI Engineer World's Fair 2025
The Robots Are Coming for Your Job, and That's Okay
Loading slides…
Read the talk
The Robots Are Coming for Your Job, and That's Okay
Twilio’s documentation workflow uses focused AI helpers to propose edits, generate metadata and simplify prose, while validation and human review keep publication under team control.
From a talk by Elmer Thomas and Maria Bermudez
Before you start: Familiarity with Markdown, pull requests and automated checks will help you follow the documentation workflow.
A small docs team facing a flood of requests
A small documentation team sits downstream of a much larger organization: Jira tickets keep arriving, first drafts need corrections, and routine grooming competes with work that requires editorial judgment. The goal for Twilio’s team is to use AI to absorb some of that repetitive work without replacing the people responsible for clarity and accuracy.
Elmer Thomas, Principal Developer Educator at Twilio, introduces the workflow alongside Maria Bermudez, a lead developer of the team’s AI Docs Buddy. Thomas describes error-prone first drafts arriving from more than 100 product teams. Style checks, image alt text and SEO metadata consume additional time. Generative AI offers help with those chores, but unconstrained generation introduces another problem: plausible content that may be wrong.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give each helper a narrow job
The team organizes the work into six single-purpose helpers behind a simple Next.js front end. Each has a bounded output that a person can inspect.
| Helper | Assigned job |
|---|---|
| Automated editor | Grammar, formatting and accuracy edits |
| Image alt text generator | Accessibility descriptions |
| Jargon simplifier | Developer language into plain English |
| SEO metadata generator | Titles and descriptions within character limits |
| Docs outline builder | Navigation and structure recommendations; coming soon |
| Slack bot | Help-channel request triage |
The outline builder is planned work at the time of the recording. The broader division of labor is already clear: repetitive jobs go to specialized helpers, leaving humans to concentrate on judgment and clarity.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn generated changes into reviewable proposals
Repeatable, high-volume work with little need for creativity is the starting point. A request enters through the Next.js UI and reaches a custom agent, with the model chosen for the particular job. The agent uses the team’s style guide and rubric, retrieved from Airtable so collaborators can maintain the guidance together. Editorial policy therefore has an explicit home outside the individual request.
The proposed changes then pass through a validation layer containing Vale linting and CI/CD tests. Vale checks configurable prose rules; it does not establish whether a product claim is true. A GitHub pull request makes the changes available for scrutiny through code-owner review. GitHub treats requesting that review and requiring approval before merge as separate settings.
Product and engineering reviewers usually participate as well, and a human merges only after accepting the changes. Generating an edit does not authorize publication. Thomas initially describes the layers as crushing hallucinations, then corrects himself: the claim is that they significantly reduce them. The protection comes from combining automated checks with several people’s judgment, rather than asking any single layer to guarantee correctness.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Inspect both the edit and its reason
Bermudez begins in the application’s overview and release notes page, which the team uses Copilot to generate. From there, she opens the automated editor. It accepts an MDX file, a Markdown file or a live URL; she chooses the URL option. This lets the editing pass start with an existing published page rather than requiring someone to prepare a separate text input.
The editor uses o1, and the interface displays a loading graphic while it works. Bermudez reports that experimentation identified it as the best model for consistently applying this team’s style guide and rubric. That choice carries a visible latency cost in the interaction.
The result offers two complementary ways to review the proposal. A diff shows what changed. The Changes Made tab provides the original text, revised text and an explanation connecting the change to a style-guide instruction or rubric item. The reviewer can inspect both the textual effect and the rule offered as its justification before deciding whether to accept it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run a separate metadata pass
The editor’s output also exposes a limit: it sometimes catches a missing SEO description and sometimes does not. Bermudez moves the same page into the dedicated SEO metadata generator. Specialization gives the missing field its own explicit task instead of relying on the general editing pass to notice every omission.
She uploads a file and returns to the Conversational Intelligence page that was just edited. The generator produces meta titles by default—users must specify if they do not want them—and a meta description that accounts for character limits. The handoff is performed by Bermudez through the interface, keeping the metadata operation separate and visible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate alt text across multiple pages
The alt text generator also supports a live URL, but Bermudez demonstrates selecting multiple pages and generating descriptions for all their images in one action. Batch input reduces the repeated work of submitting each image separately.
The results display image filenames and previews beside generated text. Bermudez calls attention to the quick completion and to a second practical requirement: the output conforms to the documentation platform’s expected format. Producing a description is only part of the task; producing one in the form the publishing workflow accepts makes the result easier to use.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make revisions easy to reuse in a pull request
The jargon simplifier serves both authors drafting a page and reviewers working through a pull request. Bermudez pastes prepared text and selects Simplify. The result includes a diff, like the automated editor, plus a separate revised-text tab. A reviewer can copy that text into a pull request comment or use it to edit the file directly.
For example, a Markdown review suggestion can make a proposed simplification directly usable:
markdown
Replace:
Utilize the configuration interface to modify notification preferences.
With:
Use settings to change your notification preferences.
The useful handoff is a concrete revision that a reviewer can accept, adjust or reject. It remains a proposal until someone applies it.
These helpers are not yet an autonomous chain. Bermudez closes the walkthrough by saying the team wants the agents to communicate with one another, but substantial work remains. The demonstrated workflow consists of focused tools with a person moving between them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Maintain quality through distinct feedback loops
Tools alone do not guarantee quality. Thomas separates the remaining risks by the kind of review they need:
- Hallucinations: Combine Vale linting and CI tests with scrutiny from human stakeholders.
- Bias: Use dataset tests and prompt audits.
- Stakeholder misalignment: Review pull requests weekly, accelerating the cycle to days or hours when needed.
These checks cover different failure modes; passing a style check does not settle a factual question or establish that stakeholders agree with the result.
Slack feedback loops with product managers and engineering provide another route for corrections. The team uses those exchanges to tune prompts continuously. Quality maintenance is therefore an ongoing collaboration around actual outputs, rather than a one-time prompt-writing exercise.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with one throughput bottleneck
The adoption process begins smaller than the full set of helpers:
- Identify one pain point that is limiting throughput.
- Choose a single task that is repeatable and rule-based.
- Loop with its users at least weekly.
Ship, measure and refine that workflow. Thomas expects several such improvements to accumulate into greater team velocity; the recording offers that as the direction of travel, without a measured team-wide gain. The practical starting point is one bounded job whose outputs people can review and whose shortcomings can feed the next iteration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A configurable prose linter for enforcing editorial rules while respecting document markup.
How to assign responsible reviewers and require code-owner approval before merging.
Updates since the talk
Reusable prompts for documentation editing, jargon simplification, alt text and SEO metadata, with examples and integration guidance.
Twilio's announcement explaining how to adopt and contribute to its public documentation prompt templates.
- Making Alt Text FastArticle
Maria Bermudez explains Twilio's batch alt-text workflow, human review and subsequently reported production results.
Full transcript
Loading transcript…