AI Engineer Code 2025

Vision: Zero Bugs

Loading slides…

Read the talk

Vision: Zero Bugs

High-assurance engineering already offers ways to prevent software defects. Coding agents could make those practices affordable far beyond aerospace and other critical systems.

From a talk by Johann Schleier-Smith

Before you start: Familiarity with functions, arrays, loops, and basic type checking is helpful; no prior formal-verification experience is required.

What would zero bugs change?

What would it mean for software to have literally zero bugs? Most people already experience something close to that impression: their phone camera, news, social media, and banking apps usually work. Engineers see the machinery behind that apparent reliability—on-call pagers, cloud outages, and the constant possibility of errors entering critical applications.

Purple Vision: Zero Bugs title slide naming Johann Schleier-Smith, Temporal Technologies, November 2025, with the speaker inset.Open full source frame
The opening slide poses the goal: software with zero bugs.

The disconnect becomes visible when an ordinary transaction goes wrong. Johann Schleier-Smith took his son to mini golf, where reservations were required and only one slot remained. He scanned the QR code and completed the booking process, only to be told someone else had taken the reservation. His son handled the disappointment remarkably well. About ten minutes later, a message revealed that the reservation had actually gone to them. They could play after all—but the software had already put them through an unnecessary emotional rollercoaster.

Schleier-Smith works at Temporal Technologies, which provides durable execution for applications running in the cloud. The connection is reliability and giving engineers time back for innovation; the zero-bugs vision extends beyond Temporal's current products. For product-specific material, he points to Cornelia Davis's Sunday workshop and Samuel Colvin's session on building agents with Temporal and Pydantic.

0:000:24
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Three reasons to doubt the goal

The first objection is practical: perhaps existing reliability is good enough. Incidents happen, people recover, and an imperfect world can tolerate some imperfect software. Many systems already achieve strong reliability where it matters most. Eliminating every remaining defect may not improve people's lives enough to justify the effort.

The second objection is technical. Systems contain millions of lines of code, and agents can increase that volume rapidly. More fundamentally, a bug is a mismatch between behavior and an end user's expectations. The user does not care whether the cause was an ambiguous product specification or a missing null check. Control systems introduce another boundary: a self-driving vehicle may encounter physical conditions its designers never modeled. Verification also has theoretical limits, including problems that become computationally intractable.

The third objection is economic. A competitor that spends less on quality may ship sooner and win the market. A minor defect with an easy workaround may cost more to eliminate than it costs users to tolerate. Schleier-Smith adds a more cynical possibility: some businesses may benefit from selling support for buggy software. Taken together, these objections make zero bugs sound like a goal that loses to complexity, competition, and incentives.

3:494:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:49 · section reference included

Engineering software as a certified component

Aerospace provides a reason to take the goal seriously. The Airbus A320's control software, developed in the 1980s, is Schleier-Smith's first example of exceptional reliability. He describes its safety record as having no serious incidents attributed to software as of the talk. That broad historical assertion is his characterization; the more actionable lesson is the combination of safeguards used to pursue reliability.

The A320 approach he describes combines several forms of assurance:

  • N-version programming: Different processors, operating systems, and implementation teams reduce dependence on a single implementation. His processor examples are Intel x86 and Motorola.
  • Specification-based design: Extensive documentation supports analysis and provable statements about behavior across different scenarios.
  • Independent verification: The people checking the implementation are separate from the people writing it.
  • Defensive programming: Allocate memory statically rather than at runtime, and handle errors simply and explicitly instead of relying on sophisticated exception handling.
  • Static analysis and verification: Examine the implementation for properties that should hold before it runs.

These techniques support a zero-defect-tolerance mindset: software is an engineered, certified component with a specification, much like a turbine fan blade.

The unit of reliability is still the whole aircraft. Software assurance must fit into protection against the many other things that can go wrong. Decades of experience with mission-critical mechanical systems offer lessons for software precisely because those disciplines treat reliability as a system property.

That makes quality through process relevant to agentic coding. Planning and requirements are familiar; external certification by a regulator or government may be less so. Integration testing is especially important when software must interact with physical equipment, a concern that grows as more software acts on the world. Feedback must improve each stage and the interfaces between stages. For an agent, such a process supplies structure for staying aligned with the intended behavior.

7:017:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:01 · section reference included

High assurance can use different architectures

The Shuttle figures Schleier-Smith cites match Charles Fishman's 1996–1997 reporting in They Write the Right Stuff: one reported error in each of three 420,000-line versions, and 17 errors across eleven versions. These are the latest versions at that reporting period, not the final releases of the Shuttle program. Schleier-Smith also says that Shuttle losses were not caused by software.

Schleier-Smith estimates that the Shuttle figures represent roughly 1,000 times fewer bugs per line than typical commercial software. Curiosity supplies a different lesson: a costly mission with little opportunity for intervention on Mars still needs exceptional reliability, but its architecture need not duplicate the A320 approach. In his account, the rover's software, developed in the 2000s, used identical redundant systems and a commercial off-the-shelf real-time operating system rather than diverse implementations and a custom operating system.

Chemical plants, automobiles, medical systems, nuclear power, and security systems offer further sources of high-assurance practice. The shared objective is dependable behavior under consequential conditions; the engineering choices can evolve with the application and available components.

11:2311:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:23 · section reference included

Make the program easier to reason about

The foundations of reliable software are also the foundations of ordinary software development. High-level languages emerged in the 1950s and 1960s, and assembly became less common as an application programming language through the 1980s. Machines increasingly generated machine code for other machines. Schleier-Smith attributes a roughly 5–10× productivity gain to the historical move to high-level languages.

The mechanism is abstraction. A programmer can work with data structures meaningful to the problem instead of manipulating memory locations directly. Preserve essential complexity and remove implementation complexity: the program must still express the business or physical problem, but it need not continually expose registers, memory layout, memory access, or machine-specific performance details. Structured programming supplies the corresponding abstraction for control flow.

Edsger Dijkstra's advocacy in the 1960s, followed by broader adoption in the 1970s, helped establish sequence, selection, and iteration as the basic building blocks of programs. Statements execute in order; if/else selects a branch; loops repeat work. Compared with arbitrary GOTO jumps through a flowchart, these structures support compositional reasoning: understand a piece, then understand how it combines with the surrounding pieces.

Structured programming does not make spaghetti code impossible. Schleier-Smith's invitation to try reading older Fortran is a reminder of how different unrestricted control flow can feel. The gain comes from hierarchical decomposition, which lets a programmer focus on one part at a time. That is just as useful when an LLM generates the code.

13:1113:23
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:11 · section reference included

Libraries still matter when code is cheap

David Parnas's work on modularity in the 1970s extends the same idea to system structure. Modularity is associated with object-oriented programming, but libraries provide it without requiring objects. A module gives a human reviewer, an LLM, or a formal verification tool a smaller unit to understand. Schleier-Smith describes the intended benefit as replacing exponential verification complexity with sub-exponential, potentially linear work through local reasoning. Achieving that benefit depends on being able to reason about the module boundaries and their composition.

Why, then, should an LLM generate a high-level language instead of machine code? Context remains scarce. An LLM may have a larger explicit context window than a person's working memory, while people carry substantial compressed background knowledge. Both still benefit from representations that concentrate attention on the relevant problem.

Cheap generation does not remove the case for libraries. Having an agent rewrite a library for one application may save little once that new implementation must be tested and verified. Trusted, separately validated components let the application inherit useful work instead of repeatedly rebuilding its assurance. Temporal's durable execution is Schleier-Smith's example: it moves responsibility for aspects of cloud reliability into a separate component, reducing what application code must handle itself.

16:5317:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:53 · section reference included

Prove an array search before running it

Dafny brings specifications and proofs into the programming language itself, and can generate code for targets including JavaScript, Python, and C#. Its verifier uses theorem-proving machinery to establish that an implementation satisfies its stated properties. The demonstration starts with an index-of method: search an array for a number and return an index. The stated conditions include a nonempty array and a result that is either -1 for absence or an index below the array's length.

A compact Dafny implementation of that search makes the contract concrete. requires constrains the input; ensures describes the result; loop invariants record what remains true as the search advances.

dafny

method IndexOf(a: array<int>, target: int) returns (r: int)
  requires a.Length > 0
  ensures -1 <= r < a.Length
  ensures r >= 0 ==> a[r] == target
  ensures r == -1 ==> forall k :: 0 <= k < a.Length ==> a[k] != target
{
  var i := 0;
  while i < a.Length
    invariant 0 <= i <= a.Length
    invariant forall k :: 0 <= k < i ==> a[k] != target
    decreases a.Length - i
  {
    if a[i] == target {
      return i;
    }
    i := i + 1;
  }
  return -1;
}

For a teaching input of [4, 9, 2] and target 9, the search passes over 4 and returns index 1. The prefix invariant explains why returning -1 after exhausting the array is valid: every earlier element has already been shown unequal to the target. The decreases clause supplies a termination measure.

The recorded sequence then exercises the verification boundary:

  1. Run the Dafny verifier on the search implementation.
  2. Generate and exercise a Python program using the functionality; verification occurs before execution in the demonstrated workflow.
  3. Introduce a small algorithm change that creates a bug.
  4. Try again and receive a verifier error.

Generating a Python library makes the verified component usable from an ordinary application. The intentional mutation shows how a failed proof obligation can stop a defective implementation before that workflow runs it.

Verification is only as good as the specification. The illustrative contract above requires a matching index, but does not explicitly require the first matching index. That distinction matters if callers expect first-occurrence behavior. A proof establishes the properties that were written down; an omitted user requirement remains an opportunity for a bug.

19:5920:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

19:59 · section reference included

From verified components to verification tools

Formal methods already support deployed systems. The seL4 microkernel is a verified operating-system kernel used in embedded and security-critical settings. Its guarantees concern the kernel and particular properties under explicit assumptions, including hardware and boot-related assumptions; they do not automatically verify every application in a deployed system. CompCert provides a compiler example: its central assurance is that compilation preserves the behavior specified by the source C program. Schleier-Smith points to security-critical and aviation applications.

Project Everest applies formal methods to cryptographic and communication components, including libraries deployed to protect internet traffic. Microprocessor design has also used formal verification for decades, driven by the high cost of incorrect hardware. These examples put verification at reusable boundaries—kernels, compilers, cryptography, and processors—where an assurance investment can benefit many downstream systems.

Schleier-Smith reports benchmark success rising from roughly 30% to nearly 100%, with runtime falling by at least 50× over roughly two decades. The talk does not identify the benchmark suite, solver versions, hardware, or task distribution, so those figures describe the progress he presents rather than a general performance guarantee. The broader mechanism is that benchmarks concentrate research effort on shared, measurable problems.

The techniques differ in what they check and how closely a proof is tied to an implementation:

TechniqueRelationship to the program
Type systems and static verificationCheck properties without running the program
Dafny and SPARKCouple specifications and proofs closely to code
Coq and LeanSupport theorem proving; implementation correspondence needs attention
Model checkingProves properties of a finite-state model

Ordinary type checking is already a basic form of static verification, and richer checks can build on it. With separately expressed proofs, the crucial obligation is showing that the proved object corresponds to the code actually used. Theorem proving can reason beyond the finite-state formulation described here for model checking.

22:3422:50
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

22:34 · section reference included

Apply assurance where the agent struggles

These practices can be useful on a specific troublesome piece of agent-generated code without imposing an aerospace process on an entire codebase. Start with detailed specifications, typed languages, and modular implementations. Then ask the model for explicit risk analysis and safety cases: what could go wrong, and what in the implementation mitigates each failure? This is qualitative reasoning, distinct from a mathematical proof, but it makes assumptions and failure handling available for inspection.

Independent verification suggests a practical division of labor. Use separate prompts for implementation and testing; for greater diversity, use different foundation-model providers for the two tasks. Add formal proofs around critical sections where the properties can be stated precisely. Keep the application small by relying on libraries that can be developed, tested, and validated separately. Each intervention reduces the amount of unchecked reasoning concentrated in one generated implementation.

26:0526:23
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

26:05 · section reference included

When the prompt itself becomes the program

Andrej Karpathy's Software 3.0 framing treats prompts as programs. An LLM might answer directly, generate code, run a loop with tools, or combine those behaviors. In that setting, the behavior being assured includes the model's own responses, not just a conventional program it emitted. Schleier-Smith argues that nondeterminism and an enormous state space prevent the verification techniques discussed so far from carrying over directly to this form of software.

The same flexibility creates a possible new form of resilience. LLMs can interpret ambiguity and respond to inputs that were not anticipated explicitly. An architecture might use agents throughout, or invoke an LLM only after a specific error condition arises. Such a system could potentially recover from cases that a rigid implementation cannot handle. That is an architectural opportunity to investigate, rather than a demonstrated guarantee of recovery.

28:0328:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

28:03 · section reference included

The cost is mostly outside the returned code

A small generated game makes the economics concrete. Schleier-Smith spent about two minutes prompting and excluded his own time from the cost calculation. The model was GPT-5-Codex. He reports 600,000 input tokens, 3.5 million cached input tokens, 48,000 reasoning tokens, and 28,000 returned tokens for the game.

Schleier-Smith reports about $2 to generate the game, with visible returned tokens accounting for about 15% of the total cost, excluding his time. The remainder went toward repeated inputs as the agent tested its work and toward reasoning. Here, “returned” distinguishes visible output from reasoning; in API billing, reasoning tokens also count as output tokens. The useful comparison with human development is that typing the final code occupies only a small portion of the work required to build the software.

30:0130:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

30:01 · section reference included

Can cheap generation pay for expensive assurance?

Schleier-Smith's cost comparison starts with high-assurance development and works down toward cheaper conventional development. These are his illustrative estimates, not a normalized comparison of projects with equivalent requirements:

Development categoryCited cost per line
Shuttle software, 1990 estimateAbout $1,000
Same estimate in 2025 dollarsAbout $2,500
Some security high-assurance softwareUp to $3,000
Typical production software, without AI$10–$100
Low-cost contractors, without AI$1–$10

The large premium buys more than writing code: high-assurance development includes the work needed to establish confidence in its behavior.

Agentic generation spans cheap models producing code with little iteration through expensive models doing substantially more work. Excluding people costs, Schleier-Smith estimates agentic generation to be at least 1,000×, and probably about 10,000×, cheaper than typical software development. He places the high-assurance premium over typical software at about 100×. These rough ratios motivate the economic argument: generation savings might be large enough to fund much more assurance work.

Schleier-Smith extrapolates that agent-produced high-assurance software could become 100× cheaper than typical software is produced today. This is a forecast from the cost ratios, not a measured result from an assurance-oriented agent. Its premise is that agents can apply the expensive verification, process, and testing practices while retaining enough of their generation-cost advantage.

31:1331:29
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

31:13 · section reference included

Fewer defects could unlock adoption

The zero-bugs vision depends on directing agents toward assurance, not merely asking them to produce more code. Schleier-Smith characterizes reliability as a solved problem in aerospace and other critical industries, and proposes agents organized around formal methods, extensive processes, and adversarial testing. The intended result is a proliferation of experiences in which users simply do not encounter defects.

This also addresses a reason developers resist coding agents: repairing the generated implementation may take longer than writing it correctly themselves. Lower defect rates can change the adoption calculation. If agents routinely produce fewer defects than human-written code, Schleier-Smith expects adoption to accelerate sharply. The assurance knowledge already exists; the opportunity is to make applying it economically routine.

The closing image is resilience with a sense of humor. Ziggy, Temporal's mascot, is a tardigrade—a member of the phylum Tardigrada, not an insect and therefore not a bug. Tardigrades can survive extraordinary conditions, including exposure to space; Schleier-Smith says Temporal took Ziggy to space earlier that year. He closes by returning to durable execution as a reliable foundation for modern software and inviting people interested in these ideas to explore working together.

33:3433:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

33:34 · section reference included

Resources

From the talk

  • Charles Fishman's December 1996/January 1997 reporting on the Shuttle software team's defect counts and engineering practices.

  • Explains how reasoning tokens consume context and contribute to billed output usage.

Transcript download
Full transcript

Loading transcript…