AI Engineer World's Fair 2025
Don’t get one-shotted: Use AI to test, review, merge, and deploy code — Tomas Reimers, Graphite
Loading slides…
Read the talk
Don’t get one-shotted: Use AI to test, review, merge, and deploy code
Getting a feature working is only the first loop. AI-assisted development also demands faster review, testing, merging, and deployment without losing scrutiny.
From a talk by Tomas Reimers
A working feature still has to ship
A developer gets the code working and the feature behaving as intended. Then comes testing, review, merging, and deployment. Faster implementation helps with the first part, but it does not remove the second. Tomas Reimers, co-founder of Graphite, an AI code review company, describes these as software development’s inner loop and outer loop.
| Loop | Work | Immediate objective |
|---|---|---|
| Inner | Code, build, debug | Get the feature working |
| Outer | Test, review, merge, deploy | Move the change through to release |
The distinction matters because producing a change and getting it safely to users are separate workloads. Accelerating one leaves the other to absorb its output.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
More generated code, more code to scrutinize
AI adoption is already changing the inner loop. Reimers points to a GitHub survey showing widespread use among surveyed developers. That evidence concerns enterprise respondents having tried AI tools; it does not establish routine use by nearly every developer worldwide, or that every respondent used them both at work and outside it. The accompanying Copilot statistic also needs its historical scope: GitHub’s February 2023 announcement reported Copilot generating an average of 46% of code for developers using it, not 46% of all code hosted on GitHub.
The forecasts that follow reinforce a directional point: even under Reimers’s more pessimistic framing, AI writes an increasing share of code. Developers consequently produce more changes, and those changes still require review.
Graphite began developing Diamond, its AI code reviewer, about a year before the talk, while its own developers were adopting AI tools. Diamond is the historical product name used here; its review capabilities later moved under Graphite Agent. The concern was not just volume. AI can hallucinate, make mistakes, and introduce security vulnerabilities, so generating code faster does not make scrutiny optional.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The outer loop needs capacity of its own
Tools such as Cursor, Windsurf, Copilot, v0, and Bolt were producing larger volumes of code than Graphite’s developers were accustomed to handling. That increased the work at every subsequent stage: reviewing, testing, merging, and deploying. The outer loop was becoming the bottleneck. Problems once associated with large engineering organizations were beginning to affect smaller companies as well.
The requirements follow the work that now has to scale:
- Pull-request coordination: Prioritize changes, track their progress, and notify people when attention is needed.
- Reviewer assistance: Help reviewers focus and streamline the review process.
- Integration throughput: Optimize CI pipelines and use merge queues to handle the increased volume of changes.
- Deployment tooling: Improve the final stage that gets accepted changes to users.
These are requirements for the wider workflow, not just for an AI reviewer. Deployment is part of that scope, though Reimers does not describe a deployment implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adding AI teammates is only part of the solution
The same technology creating pressure on review could also reduce its manual workload. Reimers’s goal for self-driving code review is to remove painful, repetitive parts of the process, leaving developers more attention for getting products to users and ensuring that features behave as expected.
But AI-generated feedback is imperfect. Graphite’s early idea—add AI teammates to the existing workflow—therefore became only part of the answer. Background agents and automated reviewers can contribute, but the tools around them must also support the increased workload. If developers become orders of magnitude more productive, as Reimers anticipates rather than measures here, the entire toolchain needs to accommodate AI-assisted development, not only the IDE.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Diamond’s place in the review workflow
Diamond is Graphite’s starting point for that broader toolchain. Reimers describes a review platform designed for high signal and low noise, with an understanding of both the codebase and its change history. Its proposed workflow combines several responsibilities:
- Change context: Summarize, prioritize, and review each change.
- Review relevance: Use codebase and change-history understanding to produce useful feedback.
- CI and testing: Integrate with existing infrastructure to summarize errors and correct failures.
The intended scope extends from understanding a pull request to helping resolve problems exposed by its checks.
As Diamond reached larger customers and enterprises, Reimers describes shorter review cycles and more consistent quality as hoped-for benefits that Graphite was beginning to observe. He also presents private and secure code handling, zero setup, one-click suggestions, and customization as product attributes. Some further expansion was not yet public. The product case returns to the broader workflow: AI should improve how changes get reviewed and shipped, as well as how they get written.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure whether review comments lead to changes
Reimers closes with reported comment-level results. Here, acceptance means that feedback was integrated into the pull request where the comment appeared. A downvote is explicit feedback on a comment, not a measured code-error rate.
| Measure | Reimers’s reported result |
|---|---|
| Diamond comments downvoted | Less than 4% |
| Human comments integrated into the pull request | Approximately 45–50% |
| Diamond comments integrated into the pull request | Approximately 52% |
The Diamond acceptance figure was new as of March and followed substantial tuning. Reimers does not supply sample sizes, measurement windows, or details establishing comparability between the human and Diamond comment groups. The useful operational distinction is between a reviewer producing feedback and that feedback resulting in a code change: the latter is the behavior Graphite was tuning for.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
- Diamond's March 2025 launchArticle
Graphite's launch announcement explains Diamond's review features and the company's broader outer-loop product vision.
A 2021 study of insecure Copilot outputs across 89 scenarios designed around high-risk security weaknesses.
A case study describing how Graphite uses developer actions, explicit feedback and evaluation datasets to improve review comments.
Updates since the talk
Current instructions for configuring review rules, exclusions and filters, and monitoring their effectiveness.
Full transcript
Loading transcript…