Why Humans Can't Keep Up With AI-Written Code - Tariq Shaukat, Sonar
Loading slides…
Read the talk
Why Humans Can't Keep Up With AI-Written Code
Tariq Shaukat examines why release counts grew far less than code volume in a study of asynchronous coding agents—and how Sonar combines intent, guide rails, independent verification and auditable evidence to build confidence into agentic software factories.
From a talk by Tariq Shaukat
At a glance
Ideas worth remembering
Code generation and software delivery are different measures. The study presented reports increases of 2,450% in lines of code and 240% in commits, but only 30% in releases.
Human approval is not sufficient evidence of independent judgment. The cognitive-surrender study described in the talk finds that people agree with incorrect AI answers four out of five times.
Repository navigation consumes real agent work. Guidance can reduce repeated discovery, while long-task capability claims must include the success threshold at which they were measured.
Sonar’s confidence formula treats intent, guide rails, independent verification and auditable evidence as jointly necessary. Better generation does not eliminate the need to verify the result or demonstrate that it passed.
Read customer metrics by their actual denominator: being more likely to report a very positive impact is different from measuring an equivalent percentage improvement in security, delivery speed or outages.
The ball bearing tests whether the parts fit
A car can have good individual parts and still fit together badly. Tariq Shaukat, Sonar’s CEO, opens with the gaps between body panels on cars of the 1980s, then recalls a Toyota/Lexus commercial in which a ball bearing runs across a car’s seams. The demonstration makes an otherwise abstract manufacturing claim tangible: the finished assembly fits together precisely. For software factories, that distinction matters. Producing components is only part of producing a dependable system.
The manufacturing lesson is to engineer quality into production rather than maximize output and repair the result afterward. That changes several decisions at once: prioritize quality over volume, correct the system that creates defects, and design parts to work together instead of hoping separately designed wheels and washer bottles will fit. Problems get addressed when they appear. Anyone can stop the line, not just the boss.
Software development already recognizes much of this logic. Shaukat connects Agile, Kanban, SRE error budgets and continuous integration to the production-system emphasis on flow and quality. His starting point is therefore familiar engineering practice. The question is what happens to that practice when agents sharply increase the amount of work entering the factory.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Agents increase output—and the pressure on the factory
Research Shaukat presents on asynchronous coding agents reports a 2,450% increase in lines of code written. Code creation becomes abundant enough to invite another step: automate the surrounding workflow and run an agentic software factory. The adoption figures he previews from Sonar’s developer survey show that ambition. Among the developers surveyed, 5% say they already run such factories; 34% expect to within a year, rising to 39% in financial services. Those latter figures describe expectations, not completed deployments.
The same capabilities also help attackers. An agent that can inspect code and find vulnerabilities can support either development or exploitation. Shaukat connects that dual use to a post by Satya Nadella about model-related insider risk: containment and governance need deterministic system design, human controls and reliable operating procedures. In this framing, confidence must come from the way the factory operates, rather than from a general belief that its models are capable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Human review faces both a throughput limit and a judgment problem
The first constraint appears when generated work must become a release. In the same study, the reported increases are 2,450% for lines of code, 240% for commits and only 30% for releases. What does that comparison reveal? Production of code has accelerated much more than delivery. The gap does not identify every downstream bottleneck, but it does show why code volume cannot stand in for shipping progress.
Adding a human approval step does not automatically solve the problem. Review capacity must handle the additional output, and the reviewer must actually challenge it. Shaukat introduces the second failure as “cognitive surrender”: in a study he describes, humans agree with correct AI answers 93% of the time, but also agree with incorrect answers four out of five times. These are agreement results from the cited study, not a measured error rate for enterprise code review; their relevance is the danger of treating human presence as proof of independent judgment.
That makes the human-in-the-loop objection stronger than a complaint about slow review. A system can have a named approver and still accept the model’s mistakes. Shaukat’s challenge to enterprise governance is to explain how the process detects bad output, rather than stopping at the reassurance that a person is involved.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Longer tasks make the success threshold consequential
The third constraint is accumulated error. METR’s task-completion time horizons measure task difficulty using the time a human would need, then associate that duration with an agent’s probability of successful completion. Shaukat describes a model reaching tasks that take humans 16 hours—but at a 50% success rate. What changes when the required success rate rises to 80%? In his reading of the chart, the task-horizon curve flattens substantially. The 16-hour figure is therefore a capability claim at a particular reliability threshold, not a promise of dependable 16-hour work.
A simple calculation explains the concern. If a workflow has 20 required steps, each succeeds independently with probability 95%, and any failed step makes the whole workflow fail, then the probability that every step succeeds is (0.95^{20} \approx 0.36). That is the roughly 36% result Shaukat gives. The independence and all-steps-required assumptions make this an illustration rather than a measured agent failure model; it shows why a high local success rate can still produce a weak end-to-end result.
For a software factory, the relevant question is whether the entire requested change succeeds at an acceptable reliability level. Extending how long an agent can work does not remove that requirement. Longer workflows give unresolved mistakes more opportunities to affect the result, which makes verification during the work consequential rather than merely a final inspection.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Another LLM is not automatically a reliable judge
The fourth constraint challenges an obvious response to review overload: ask another LLM to judge the output. Shaukat presents two different forms of instability:
- Repeated judgments: Research he attributes to PayPal AI’s lab reports that 7–17% of verdicts change across identical replications, depending on the model and situation. Repeating the same question can produce a different decision.
- Challenged judgments: Research he attributes to Meta reports that a simple “Are you sure?” challenge changes the model’s answer 7–34% of the time, again depending on the model.
Neither percentage establishes that every changed verdict is wrong. The operational problem is that a repeated check or an expression of doubt can alter the decision without supplying new evidence about the code. An LLM judge may contribute to verification, but its presence alone does not deliver the consistency that a release decision needs.
The fifth constraint is the shrinking time available to respond to exploitation. Shaukat characterizes time to exploit as nearing zero, without developing a numerical benchmark in this talk. His design conclusion is prevention: when the response window is extremely short, the factory needs to detect and prevent problems before they become something an attacker can use.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Four requirements for code confidence
Sonar’s response is the code confidence formula: (\text{confidence} = \text{intent} \times \text{guide rails} \times \text{verification} \times \text{evidence}). It is a design model, not a calibrated numerical score. Multiplication expresses the judgment that a missing requirement can invalidate confidence in the whole factory, even if the other parts look strong.
Each factor answers a different question:
- Intent — What are we trying to build? An agent cannot reliably deliver the desired outcome if the organization cannot state it clearly.
- Guide rails — What keeps the work on course? Shaukat deliberately distinguishes these from guard rails. Guard rails stop something going off the road; guide rails supply smaller nudges that help it stay on the road while working.
- Verification — Has the result been checked independently and thoroughly? Findings must lead to fixes, rather than being deferred until the end or passed downstream.
- Evidence — Can we demonstrate that the checks happened? The proof should be consistent, auditable and repeatable.
The evidence requirement changes what counts as completion. A team may believe it specified, guided and checked a change, but without proof it cannot reliably explain that conclusion to customers, executives or a board. The factory must produce both the software outcome and the basis for accepting it. Sonar uses these four requirements to organize the systems it builds with customers and partners.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Guidance reduces search work before independent verification
Sonar Vortex applies intent and guide rails inside the agentic loop, with verification that nudges the agent as it works. For long-running tasks in large, complex codebases, the proposed mechanism returns to White Plains: give the agent information that places it closer to the answer. It should not spend tokens rediscovering things the development system already knows.
What changes in cost and output when that guidance is present? Shaukat reports customer results of a 36% reduction in token cost and a 92% reduction in issues in the commit. The two measures address different consequences: less work spent finding the way, and fewer issues entering the subsequent verification stage. These are Sonar-reported results for the described setting; the talk does not specify sample sizes, comparison protocols or how commit issues were counted.
The ordering matters. In-loop guidance improves what arrives at independent verification; it does not make that later check unnecessary. This is the software version of correcting the production process before the finished assembly reaches inspection.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Verification must be affordable, and its results must be provable
Independent verification has its own engineering tradeoff: cover the relevant problems precisely without making every check unnecessarily expensive. Shaukat’s rule is to use CPUs where CPUs suffice and GPUs where GPUs are needed. Sonar combines a deterministic mixture of methods with agentic methods in SonarQube. The important distinction is that verification can draw on different kinds of checks, rather than delegate every decision to an LLM judge.
The customer-survey comparison needs a careful reading. Shaukat describes recent conversations with more than 1,000 customers and presents SonarQube users as more likely than non-users to report a very positive impact from AI coding. How do the three percentages relate to the outcomes? They compare the likelihood of reporting a benefit, rather than directly measuring how much security, delivery speed or outage frequency changed.
- Code security: SonarQube users were 81% more likely to report a very positive impact.
- Time to market: Users were 34% more likely to report a very positive impact.
- Outage frequency: Users were 76% more likely to report a very positive impact.
These are associations in reported experience, not evidence of an 81% reduction in security issues, a 34% acceleration in delivery or a 76% reduction in outages. Nor does the comparison by itself isolate verification as the cause.
The final requirement is to retain evidence that the change passed the applicable quality gate and controls. Shaukat names IT general controls for internal operations and MISRA for automotive work as examples of different obligations a factory may need to satisfy. A reassuring result is not enough: the organization needs proof that it met the requirements relevant to its software.
That brings the factory analogy back to its practical purpose. More generated code is useful only if the system can turn it into outcomes the organization can confidently accept and ship. Shaukat closes with the distinction “code is abundant” and “confidence is scarce.” The proposed factory produces that scarce confidence through clear intent, guidance during generation, independent checks, fixes and evidence—not through a larger pile of code awaiting human approval.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Charts and methodology for understanding task difficulty in human-time units and why a task-horizon claim depends on its required probability of success.
Further reading
Product background on the guidance tool discussed in the talk, with additional context for repository guidance and reported token savings. Its benchmark conditions should be kept separate from this recording’s customer figures.
Related talks
- In the Land of AI Agents, the Verifiers Are King
Shaukat develops the complementary Guide, Verify, Solve framework, including the distinct roles of in-loop verification, CI verification and continuing code maintenance.
- Fixing the PR Bottleneck — Matt Pocock, AIHero
Adds concrete examples of misleading green tests and explains how deterministic checks, automated review and risk-focused human review can share the growing review workload.
- Building agents is trivial now, context is the next frontier
Extends the navigation problem beyond repository structure: an agent’s plausible recommendation can still be wrong when it lacks the organizational history behind the code.
Full transcript
Loading transcript…