Contents
  1. Foundations and history
    1. What a reviewer can accept
    2. Reuse and organized production
    3. Delivery pipelines and learned generation
  2. Work and responsibilities
    1. Work intake and admission
    2. Specifications and acceptance criteria
    3. Execution and acceptance
    4. Dividing work for recombination
  3. Evidence and acceptance
    1. Artifact and evidence identity
    2. Checks and acceptance claims
    3. The combined candidate
  4. Release and learning
    1. Release authority and delivery confirmation
    2. Exception ownership and recovery
    3. Improving the production process
  5. Capacity and design
    1. Throughput and admission limits
    2. Comparing factory boundaries
  6. Reference questions
    1. Frequently asked questions
      1. How does a software factory relate to DevOps and CI/CD?
      2. Does a factory require several agents?
      3. Can work start with incomplete requirements?
      4. What do passing tests establish?
      5. Which changes can run unattended?
      6. Who owns a bad release?
      7. Does more generated code demonstrate value?
  7. Check understanding
  8. Open questions
  9. Selected talks
  10. References
  11. Talk library
← All topics

Software factories

A software factory is a production system that coordinates people, tools, and automation to turn software requests into delivered changes through a repeatable process. In his World’s Fair 2025 demonstration, Eno Reyes, cofounder of Factory, asked Code Droid to retry a Firecrawl scrape when it returned no content. Droid returned a pull request for a maintainer to review. Zach Lloyd’s proposed workflow at Warp shows where that handoff fits. After reviewing the code, someone still has to try the feature, decide whether to ship it, and observe how it behaves in production.

Foundations and history

What a reviewer can accept

Reyes’s request to Code Droid named the repository, the failure and a possible delay: “In firecrawl, if we get a no content error - retry with a waitfor of maybe 2000ms more in factory-mono”. The error tells an engineer when the retry should happen. The word maybe leaves the delay open to discussion. Code Droid can inspect the repository before it proposes the change. The coding-agent execution loop

Factory home screen with a request to retry a failed content scrape.
Factory demo: submitting a task — Ship Production Software in Minutes, Not Months · Eno Reyes · Watch at 3:48

Droid inspected the checkout, identified the branch, and consulted recent changes before proposing a plan and asking Reyes for clarification. As Reyes put it, “AI tools are only as good as the context that they receive.” The repository told Droid how the code worked. Reyes could clarify what he wanted it to do. After his reply, Droid ran checks and prepared a pull request (PR), a proposal to incorporate its change into the shared codebase. Reyes reported that the PR passed the project’s automated continuous integration (CI) checks.

A patch and a test report are artifacts, outputs a receiver can inspect. A maintainer can read the patch to see which error should trigger a retry. To confirm that the retry ran, the maintainer would need a test result or trace showing a no-content response followed by a second scrape. Confirming the fix in production would also require identifying the deployed version and observing how it handled that condition. Reyes’s demonstration ended at the PR.

In Lloyd’s proposed workflow, monitoring can uncover a bug for triage, where the team decides how to handle the report. The team chooses which repositories and stages to automate. Lloyd’s diagram marks three points where people review the work. View the original diagram at full resolution · Lloyd’s interview on factory design

  • Reviewing the spec — Settle what the feature should do before implementation begins.
  • Reviewing the code — Inspect the proposed change against the agreed behavior.
  • Reviewing the product — Try the working feature and decide whether it is ready to ship.
Circular software lifecycle diagram with triage, spec, implementation, verification, shipping, and monitoring, plus human review points.
A cloud software factory — Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp · Zach Lloyd · Watch at 3:21
These three examples from 2025–2026 illustrate different review decisions. Alongside Reyes’s retry task and Lloyd’s proposed workflow, Uber illustrates how a reviewer could assess a stadium pickup feature in its rider app. Uber’s walkthrough
AccountWhat starts workWhat gets checkedHuman decision
Factory · demonstrated taskA request to retry a scrape that returns no contentDroid runs project checks on the patch.Reyes clarifies the plan. A maintainer can then assess the proposed retry.
Warp · proposed workflowA product idea or bug reportSpecification, implementation, and working productSettle intent, review the code, and decide whether the product is suitable to ship.
Uber · illustrated featureA pickup idea discussed in SlackMinion, Uber’s coding agent, attaches check results and a simulator screenshot to a draft PR.Compare the rendered screen with the design and decide whether the draft needs more work.

The July 2026 World’s Fair Software Factories track brought together different questions about this loop. The program listed Factory’s Tereza Tížková under Rise of the Software Factory. Her recorded talk is What It Actually Takes to Build a Software Factory. Lloyd’s session on self-improving factories was released as Software Engineering Is Becoming Factory Engineering. Dex Horthy of HumanLayer examined the review and testing burden in Why Software Factories Fail. These concerns also apply to a team using one agent, ordinary build tools, and human maintainers. The scope of the factory follows the work those participants undertake together. World’s Fair program · Defining completed work

Reuse and organized production

Software was easy to copy long before it was easy to produce. At the October 1968 NATO Software Engineering Conference, Douglas McIlroy asked why programmers kept constructing familiar routines when a supplier could offer dependable families of them. A programmer might need the same operation at a different precision or with a different tradeoff between memory and execution time. The supplier’s contribution would be to engineer, describe, and maintain those useful variants. A catalogue of identical copies would solve little of that problem. McIlroy’s component proposal, section 8.2

For organizations taking on large contracts, the scarce resource could instead be a dependable way of organizing people. In 1969 Hitachi separated its Software Works from its Kanagawa Works as orders grew for systems including railway reservations and online banking. System Development Corporation (SDC) tried a division of labor in 1976. Managers at customer sites retained responsibility for system design. A central factory took on detailed design, programming, and testing with shared procedures and tools. Michael Cusumano’s historical study describes the difficulty of directing work across those management lines while contract demand fluctuated. SDC ended the experiment in 1978, although standardized engineering practices survived. Sharing a method across projects proved possible even when pooling the work in one organization proved difficult. Hitachi’s history · Cusumano’s SDC study

A software product line makes a third choice about reuse. Its designers identify related products, implement what they share, and plan where one product can vary from another. In their 2003 Software Factories proposal, Jack Greenfield and Keith Short used domain-specific languages to express choices in terms familiar to people working in a particular application area. A template brought reusable implementation assets and development guidance into the programming environment. This approach requires someone to understand the product family well enough to anticipate useful variations. It gives the next developer a way to express those variations without rebuilding the common parts. Greenfield and Short’s proposal

Delivery pipelines and learned generation

Even a team making a single product has a recurring production problem. An edit that works in one developer’s checkout must still work alongside everyone else’s changes. Martin Fowler’s account of continuous integration (CI), first published in 2000, describes combining work at least daily and checking each integration with an automated build and tests. Continuous delivery extends those checks until the team can release a version on demand. Continuous deployment goes further by automatically releasing changes that pass the pipeline. Fowler on continuous integration

Fragile manual deployments at ThoughtWorks motivated the deployment pipeline documented in 2006. Its practical insight was to build a package once and promote that package through successive assessments. A staging test then describes the same package that production can receive. Environment differences still matter, but rebuilding between stages introduces a needless uncertainty about what was tested. DevOps also addressed the working relationship between those developing software and those operating it. Patrick Debois founded devopsdays, which held its first event in Ghent in 2009. Testing, automation, and organizational culture are recurring subjects in the conference series. Jez Humble on deployment pipelines · History of devopsdays

Learned generation changed the source of a proposed implementation. In the July 2021 Codex study, a model trained on public code generated Python functions from docstrings. Executable tests assessed each function on a specified task. In May 2025, GitHub introduced a Copilot coding agent that could take a repository issue, inspect the code in a GitHub Actions environment, push commits to its own branch, and request review on a draft PR. At launch, a human had to approve running the PR’s CI/CD workflows. The unit of delegation had expanded from writing a function to preparing a repository change. Existing review and integration decisions still applied. Codex study · GitHub’s launch announcement · Earlier approaches to automated change

Work and responsibilities

Work intake and admission

Work intake covers receiving requests, clarifying what they mean, ordering them by priority, and selecting work to start. Admission is the point at which the team commits capacity and an owner to a result. A ticket can be important and still lack the information needed to admit it. Lloyd distinguishes a straightforward issue that can go directly to implementation from one that needs a product or technical specification. For a disputed architecture, the useful first assignment might be a comparison of two options for the person authorized to choose. That investigation has a finish condition of its own. Warp’s triage workflow · Transferring responsibility

Readiness includes the means to finish. A dependency update may have a known scope and an agreed compatibility test, yet need repository access, a running consumer service, or time from its maintainer. Record which prerequisite is missing. Deferring that update leaves its priority intact and keeps a work slot available for something that can finish. In the intake diagram, clarification changes what the team knows about a request. Deferral changes when the team undertakes it. The Kanban Guide on selecting work

Which requests become commitments?

Example

Settling intent and making capacity available resolve different reasons for waiting.

Received request
DeclineRecord the disposition.
Clarify the outcome
Refer to the decision owner
Check readiness and capacityConfirm access, dependencies, checks, and a receiver.
Defer with a reason
Admit the workAssign responsibility for the agreed result.
In this example policy, an out-of-scope request is declined, an unclear request goes back for clarification, and a decision outside the team’s authority goes to its owner. A bounded request is admitted only when prerequisites and capacity are available.
Read the diagram as text
  • Received request.
  • Decline. Record the disposition.
  • Clarify the outcome.
  • Refer to the decision owner.
  • Check readiness and capacity. Confirm access, dependencies, checks, and a receiver.
  • Defer with a reason.
  • Admit the work. Assign responsibility for the agreed result.
  • Received request → Decline: Outside mandate.
  • Received request → Clarify the outcome: Intent unclear.
  • Received request → Refer to the decision owner: Decision exceeds authority.
  • Received request → Check readiness and capacity: Wanted and bounded.
  • Check readiness and capacity → Defer with a reason: Prerequisite or capacity missing.
  • Check readiness and capacity → Admit the work: Ready and capacity available.

Specifications and acceptance criteria

A specification records the behavior and constraints people have agreed on. It identifies the effect wanted, existing guarantees to preserve, relevant operating limits, and changes outside the assignment. It can leave the implementation open. An acceptance criterion states an observable condition a receiver can use to judge the result. Recording why a condition matters helps distinguish a necessary guarantee from an implementation preference. An open decision that could change what users receive needs an owner. Defining technical requirements · Specifying what an operation promises

Suppose an export API gains an option to include a timezone. Callers that omit the option must continue to receive the original fields and timestamp values. The developer can choose the serializer. For callers requesting the new field, there is a product decision: should an export use the account’s zone or the requester’s? Either value can satisfy a schema requiring a timezone string. Once the owner chooses the account zone, a revealing acceptance example is a Paris-based requester exporting a Tokyo account. Expect Asia/Tokyo in the new field. Run the same export without the option to check the separate preservation promise.

Prose can preserve the reason for choosing the account zone. A schema specifies the representation. A scenario, such as one written in Cucumber’s Gherkin language, connects a starting state and action to an observable result. These support verification, assessing conformity to the agreement. Validation asks whether the export serves its recipient’s intended use. A shared Definition of Done adds quality obligations across work items, such as documenting changes to public response fields. Gherkin reference · Verification and validation · The Scrum Guide

Tížková’s slide describes Droid’s plan for a 750 mm wall-mounted rotor. It divides the design into a 3×3 grid so each tile fits the printer bed. Droid produced the tile and joinery files and suggested pocketing the back. Checking the tile dimensions addresses one acceptance criterion. To assess the assembled fit, a reviewer would need evidence from joining the printed tiles.

A rotor logo design and 3D printer illustrate an agent-assisted task that generated printable tiles and joinery.
Example of the loop — What It Actually Takes to Build a Software Factory — Tereza Tížková, Factory · Tereza Tížková · Watch at 10:42

Execution and acceptance

Design a stage by working backward from the receiver’s decision. What input does the executor need, what may it change, and what must it hand over? A harness gives an agent an environment and tools in which to act. A gate enforces a condition for advancing the resulting work. Access to a test runner permits an agent to gather evidence. The organization separately decides who can accept a contribution, expose it to users, or grant an exception. Those decisions can be delegated to automation under a policy with an accountable owner. Agent Runtimes and Harness Engineering · Allocating work and authority separately

One possible division of responsibilities for a feature. A small team can combine roles while keeping the decisions explicit.
Stage and inputWho produces whatWho decides what happens next
Investigate a requestAn engineer or agent identifies affected behavior and open choices.The requirement owner settles the scope or requests more investigation.
Implement agreed behaviorA worker prepares a patch against a known revision.A maintainer takes that revision for assessment.
Check a candidateTools report observations. Reviewers compare them with expectations.The maintainer accepts the evidence, asks for another check, or returns the patch.
Integrate reviewed changesA merge queue builds and checks a proposed shared revision.Repository policy permits it to land or sends a failed combination for investigation.
Deliver a packageRelease automation installs an identified build and records its operating signals.The release owner or delegated policy expands, pauses, or stops exposure.

In Uday Kiran Medisetty and Adam Huda’s stadium pickup walkthrough, Uber’s assistant Cortana helps turn a Slack discussion into requirements and two possible button designs. Minion takes over implementation. The build produces a pull request. During validation, Minion can still revise that draft before a person reviews it or shared CI runs. Comparing a simulator screenshot with the Figma design lets a reviewer judge the button’s appearance. Checking which riders see the button also requires exercising the condition that makes it appear. The demonstrated handoff is the draft and its check results. Uber’s walkthrough · Giving ownership real authority

Dividing work for recombination

A good division of work leaves each receiver with something small enough to understand and a way to check it. A dependency exists when a contribution requires another’s particular output or interface. Consider a service with web and command-line clients. Once their response format is agreed, all three implementations can proceed independently against it. The clients do not need each other’s code. Both eventually need to read a real service response. The diagram shows why drawing the interface early creates room for concurrency and creates a specific obligation at integration. Decomposing work for recombination

In Robert Brennan’s OpenHands refactor workflow, imports reveal dependencies between groups of files. A worker can start with a batch whose prerequisites are complete and submit a reviewable change to an integration branch. Later workers incorporate upstream changes, and maintainers can assess the migration as a whole before proposing it for the main branch. Smaller batches reduce the amount of code in any one review. Splitting further eventually adds handoffs faster than it simplifies those reviews. Brennan’s demonstration · OpenHands integration workflow

Separate clients share a service contract

Example

The clients can be built concurrently, but each owes a compatibility check with the producer.

Agreed response format
Web client
Service
Command-line client
Check web display
Check command-line output
An agreed format guides three implementations. The web client and command-line client have no dependency on each other. Each integration check takes its own reader and the actual service as inputs.
Read the diagram as text
  • Agreed response format.
  • Web client.
  • Service.
  • Command-line client.
  • Check web display.
  • Check command-line output.
  • Agreed response format → Web client: Fields to render.
  • Agreed response format → Service: Fields to produce.
  • Agreed response format → Command-line client: Fields to parse.
  • Web client → Check web display: Web reader.
  • Service → Check web display: Actual response.
  • Service → Check command-line output: Actual response.
  • Command-line client → Check command-line output: Command-line reader.

Multi-agent coordination distributes work and decisions across several agents. Separate Git worktrees give each writer its own checked-out files, but writers can still make incompatible design choices or affect the same external resource. In Luke Alvoeiro’s account of Factory Missions, one worker or validator at a time acts on a feature. Searches, research, and read-only reviews can run concurrently. That arrangement preserves concurrency where findings can be combined cheaply and serializes edits where one feature depends on the application left by its predecessor. Git worktrees · Alvoeiro on the Missions architecture

Evidence and acceptance

Artifact and evidence identity

A release operator finds a passing test report and a package ready to deploy. How can the operator tell whether they belong together? Start with the package’s identity, such as a digest computed from its contents. Build provenance records how an identified output was produced. The Supply-chain Levels for Software Artifacts (SLSA) specification connects the output to a builder, build definition, execution, and recorded inputs. A source revision records what the builder obtained from a repository even if the branch later moves. The statement depends on trust in the builder. Behavioral evidence comes from exercising the package. SLSA v1.2 build provenance · Identifying what was tested and released

Two paths meet at the behavioral report in the figure. The test run records which package ran, under which configuration and conditions, and what happened. The requirement supplies the expectation against which to judge that observation. If the requirement changes, the receiver can sometimes reassess the recorded result. If the new requirement concerns behavior the test never exercised, another run is needed. A dependency change can also require a new build and a compatibility check on that build. Keeping the links among requirements, software versions, and checks is called traceability. Requirements management and traceability

One package, two kinds of evidence

Example

Only the behavioral verdict depends on what the requirement says the package should do.

Source and build inputsRevision, build definition, and recorded dependencies.
PackageIdentified by a digest of its contents.
Builder’s provenance statement
Test environment and configuration
Test run
Agreed requirement
Behavioral report
Build provenance identifies the package and its recorded build inputs. A separate report links that package to an observation in a test environment and judges it against an expectation. Changing the expectation can change the verdict without changing the build or the observation.
Read the diagram as text
  • Source and build inputs. Revision, build definition, and recorded dependencies.
  • Package. Identified by a digest of its contents.
  • Builder’s provenance statement.
  • Test environment and configuration.
  • Test run.
  • Agreed requirement.
  • Behavioral report.
  • Source and build inputs → Package: Builder produces package.
  • Source and build inputs → Builder’s provenance statement: Inputs named.
  • Package → Builder’s provenance statement: Output digest.
  • Package → Test run: Candidate exercised.
  • Test environment and configuration → Test run: Execution conditions.
  • Test run → Behavioral report: Observation and candidate identity.
  • Agreed requirement → Behavioral report: Expectation for verdict.

A handoff connects those records to the assignment. Give the next owner the work item and agreed specification, the source baseline and patch, the package and configuration tested, the reports, and the obligations still open. Missions workers record commands, exit codes, discovered issues, and unfinished work before their sessions end. These notes tell the receiver where to resume. Retaining the actual reports alongside them lets someone inspect what happened after a temporary CI log expires. Retaining execution artifacts

Checks and acceptance claims

A test oracle is the rule that decides whether an observed result is acceptable. If a developer and a test both interpret an exclusive limit as inclusive, the test can exercise the exact boundary value and pass the wrong behavior. In a 2015 study of automated program repair, researchers assessed patches using tests that the repair tools had never seen. Some patches passed every test available during construction and still broke desired behavior in the independent cases. The same distinction matters when an agent writes its own tests or a second agent reviews the change using the same assumptions. Independent tests in automated repair · Assessment independence

Choose evidence by the mistake it could expose. Static analysis can flag an incompatible type. A behavior test can show that a supported input still works. A dependency scan can identify a known vulnerability for someone to assess in its deployment context. A browser check can show whether a person can complete a flow across several services. A domain reviewer can judge whether that flow solves the intended problem. In Missions, the planner assigns assertions to features before implementation. Validators can then use the agreed assertions, code checks, and live application flows to challenge what the worker built. Matching checks to claims

Consider a patch that changes a failing assertion so that it passes. The change may be legitimate if the requirement changed. A receiver needs the new agreement and separate authority to approve the altered check. Keep enforcement outside the implementer’s unrestricted control. GitHub can dismiss approvals when a diff changes or require approval of the latest reviewable push by someone other than the person who made that push. Inspect the configured bypass permissions and check reports as well. A status accepted by branch protection can be skipped or neutral, so the green merge indicator alone cannot establish that the intended test ran. GitHub protected branches · AI Security

The combined candidate

Integration brings contributions into the intended shared version and checks how they interact. A textual conflict occurs when the version-control system cannot automatically reconcile the edits. A semantic conflict can pass through a clean merge. Suppose one patch reverses a service’s default list order to oldest first. Another patch adds a client that labels the first entry as the latest. The service author tests the new ordering. The client author tests against the original service, which returned the newest item first. Only when the two patches meet does the client label the oldest item as the latest.

Notice that the two branch reports in the figure keep their original subjects. The combination introduces a third subject needing its own report. If a combined failure is intermittent, replay the same operation from reset data on the baseline, each individual change, and the combination. Repeat under matched conditions. A failure reproduced with one change alone narrows the investigation. A failure observed only with both changes points toward their interaction. Substituting a unit test for one of these replays leaves open what that revision would do in the integrated scenario. Verifying the combined result

Where the wrong entry first becomes visible

Example

Only the combined candidate contains both the new default order and the new reader.

1 / 2 · Check separate revisions

Each passing observation belongs to a different version derived from the common baseline.

New this stepBaseline · newest first
New this stepA · reverse default order
New this stepB · show latest entry
New this stepA report · oldest firstMatches the new ordering criterion.
New this stepB report · latest displayedMatches the expected client behavior on the baseline service.
In this example, the baseline returns the newest entry first. The service patch reverses the order. The client patch displays the first entry as the latest. Separate checks pass on their respective revisions, but the combined client displays the oldest entry under the latest label.
Read the diagram as text
  • Baseline · newest first.
  • A · reverse default order.
  • B · show latest entry.
  • A report · oldest first. Matches the new ordering criterion.
  • B report · latest displayed. Matches the expected client behavior on the baseline service.
  • A and B · combined revision.
  • Combined report · wrong entry. The oldest item appears under the latest label.
  • Baseline · newest first → A · reverse default order: Change service order.
  • Baseline · newest first → B · show latest entry: Add first-entry reader.
  • A · reverse default order → A report · oldest first: Exercise service.
  • B · show latest entry → B report · latest displayed: Exercise client on original service.
  • A · reverse default order → A and B · combined revision: Changed producer.
  • B · show latest entry → A and B · combined revision: New consumer.
  • A and B · combined revision → Combined report · wrong entry: Exercise combined behavior.
  1. Check separate revisions. Each passing observation belongs to a different version derived from the common baseline. Active: Baseline · newest first, A · reverse default order, B · show latest entry, A report · oldest first, B report · latest displayed. New: Baseline · newest first, A · reverse default order, B · show latest entry, A report · oldest first, B report · latest displayed.
  2. Check the combination. The same client operation now reads a list ordered in the opposite direction. The earlier reports keep their original meaning. Active: Baseline · newest first, A · reverse default order, B · show latest entry, A report · oldest first, B report · latest displayed, A and B · combined revision, Combined report · wrong entry. New: A and B · combined revision, Combined report · wrong entry.

Frequent integration shortens the interval during which a branch can diverge. A merge queue handles a target that moves while work awaits acceptance. GitHub constructs a temporary merge group containing the PR, the latest target, and changes ahead of it. An Actions workflow that supplies required checks must handle the merge_group event to assess that combined commit. If a contribution leaves the group or the base changes, establish which interactions the new candidate introduces before relying on an earlier result. GitHub merge queues

Uber’s SubmitQueue builds likely combinations before it knows which ones will land. This can reduce waiting, but a discarded combination consumes build capacity. In a 21-week before-and-after observation spanning a July 2024 scheduling change, Uber reported that its Go repository’s mean weekly builds per change fell from 3.39 to 1.85. The authors tracked waiting time alongside build consumption because deferring every speculative build could save compute while delaying integration. The measurement ends at integration. SubmitQueue study · Gautam Korlam on combined builds

Release and learning

Release authority and delivery confirmation

Deployment installs a version. Release makes its behavior available to users. A release owner can authorize exposure, stop expansion, and arrange recovery. In Google’s account of Rapid, a project blueprint identified owners, build and test targets, and deployment rules. Authorized automation could promote a particular package according to the service’s policy. Building once makes the package traceable across environments. Checking installed identity and configuration tells the operator which version is actually serving requests. Google’s release engineering account · Deployment pipeline patterns

During a canary rollout, the candidate serves a small share of requests. Its error rate can rise sharply while the fleet average barely moves because most requests still reach the existing version. Canarying releases · Limiting exposure

Example canary architecture. Authorization binds candidate C and its target; the deployment record identifies an attempted installation. Version-separated observations inform exposure decisions but do not guarantee experimental isolation. Representative traffic and shared dependencies still matter; stopping does not guarantee exact restoration.

How a canary informs exposure

Version-separated observations reveal differences that can be hidden in a fleet average.

Production traffic
Candidate version
Control version
Assess rollout criteria
Expand exposure
Hold exposure steady
Halt and recover
A bounded share of traffic exercises the candidate while existing software serves control traffic. Compare version-specific results. Expand when representative evidence meets the criteria, hold exposure steady when the comparison is inconclusive, and halt for an unacceptable regression.
Read the diagram as text
  • Production traffic.
  • Candidate version.
  • Control version.
  • Assess rollout criteria.
  • Expand exposure.
  • Hold exposure steady.
  • Halt and recover.
  • Production traffic → Candidate version: Limited requests.
  • Production traffic → Control version: Control requests.
  • Candidate version → Assess rollout criteria: Candidate errors and latency.
  • Control version → Assess rollout criteria: Control errors and latency.
  • Assess rollout criteria → Expand exposure: Representative results meet criteria.
  • Assess rollout criteria → Hold exposure steady: Sparse or confounded evidence.
  • Assess rollout criteria → Halt and recover: Unacceptable regression.

Operational acceptance transfers continuing responsibility to people equipped to exercise it. Google’s site reliability engineering teams examine dependencies, monitoring, capacity, and emergency procedures before taking on a service, sometimes after launch. Training and access make the handoff usable. Recovery also includes effects outside the software version. Restoring an earlier package can prevent another erroneous message from being sent. Someone still has to decide what to do for the customers who received the first one. Production readiness and onboarding · Restoring software after side effects

Exception ownership and recovery

An exception is work that needs a remedy or decision outside its normal path. Preserve the work item, candidate or operation identifier, completed steps, observations, and remaining obligation. Assign it to someone who can change the blocking condition. Specify when that person must acknowledge it and what would permit work to resume. Bound repair attempts by time or count, and stop earlier if repeated attempts produce the same failure without new information. Acknowledgment matters: in PagerDuty, notifying a responder, claiming an open incident, and resolving it are three different events. An unacknowledged incident can continue up the escalation path. PagerDuty incident lifecycle · Operating an exception queue

An example routing policy groups blocked work by the remedy required.
Blocking conditionReceiving ownerBasis for resuming or closing
Competing interpretations of intentRequirement ownerA recorded choice and updated acceptance criterion
Candidate fails a behavior checkImplementerA corrected revision and a rerun of the failed check
Test environment unavailableEnvironment operatorA working environment, followed by a completed check on the candidate
Producer and consumer disagreeInterface maintainersAn agreed contract and evidence from a compatible combination
Action exceeds delegated permissionOwner authorized to decideScoped authorization, a revised assignment, or a refusal
Deployment request times outRelease operatorThe original operation reconciled with the observed running state

A timeout means the caller stopped waiting. The remote system may still have installed the package. Look up the original operation and inspect the running version before submitting another deployment. If it completed, resume verification there. If it partly completed, preserve what succeeded and identify the remaining effects. Compensation is a new action that remedies a completed effect while respecting changes that have happened since. Recovery may itself need to be resumed, so retain its progress too. Uncertain external outcomes · Compensating transactions

Improving the production process

Repairing a candidate resolves one assignment. Changing how subsequent assignments are selected, implemented, or assessed requires a diagnosis of the production process. A rejected patch may reveal a missed requirement. An escaped defect may reveal that the integration environment omitted a dependency. A successful delivery can help identify the conditions under which a check was informative. In each case, preserve the observation, test the proposed explanation, and give someone responsibility for the resulting change. Postmortems and owned follow-up · Observability · Testing a failure explanation

Uber’s uReview illustrates two uses of a review finding. A developer can act on a comment to fix a PR. The team operating the reviewer can use ratings and comment metadata to adjust a filter, prompt, or model. A developer’s usefulness rating tells the team something about a comment that appeared. It says little about an issue the reviewer missed. Uber also benchmarks against changes with human-annotated issues. Its report identifies a practical limit: source code alone may omit a configuration or schema needed to understand system behavior. Uber’s uReview engineering report

Testing a repair and testing a reviewer

Example

The case that prompted a revision and the cases reserved to judge it answer different questions.

Investigated failure
Reserved representative changes
Original reviewer
Revised reviewer
Known-case check
Paired assessmentApply the same expectations to findings from both reviewers.
An investigated failure guides a reviewer change and supplies a known regression case. Separate representative changes go through both reviewer versions. Comparing their findings against common expectations tests whether the revision helps beyond the case used to develop it.
Read the diagram as text
  • Investigated failure.
  • Reserved representative changes.
  • Original reviewer.
  • Revised reviewer.
  • Known-case check.
  • Paired assessment. Apply the same expectations to findings from both reviewers.
  • Investigated failure → Revised reviewer: Diagnosis guides revision.
  • Investigated failure → Known-case check: Case to reproduce.
  • Revised reviewer → Known-case check: Revised finding on known case.
  • Reserved representative changes → Original reviewer: Evaluate reserved changes.
  • Reserved representative changes → Revised reviewer: Evaluate identical changes.
  • Original reviewer → Paired assessment: Original findings on reserved work.
  • Revised reviewer → Paired assessment: Revised findings on reserved work.

When changing the reviewer, keep the motivating failure as a regression case and check behaviors that worked before. To test a claim about future work, reserve fresh representative cases from the revision process. This reserved sample is a holdout. Run the original and revised configurations on the same cases under the same criteria, preserving both versions and their findings. Once a score influences the next prompt or model choice, the cases behind it have helped shape that choice even if nobody saw the individual answers. Use fresh evidence for the next independent comparison. Holdout reuse · Revalidating a changed system

Capacity and design

Throughput and admission limits

Choose a work item and a finish event before measuring a factory. Throughput counts distinct accepted completions in a stated period. Lead time measures elapsed time from a stated starting point to that finish, including waiting. Work in progress counts admitted items still unfinished. Track their age as well as the duration of completed work. A fast stream of easy completions can conceal one long-blocked item. Multiple attempts on that item consume resources while leaving its completion count unchanged. Flow measures in the Kanban Guide · Measuring the complete process

Dex Horthy describes how review can limit delivery as agents produce more patches. Compare two possible workloads for the same day. In each, the queue starts empty and a reviewer has time for four candidates. All candidates take the same time to review, arrive in time for a slot, and pass on the first attempt. Count a candidate as finished when the reviewer accepts it.
Candidates arrivingReview slotsAccepted that dayLeft waiting
4440
8444

The four candidates left waiting become part of the next day’s workload. A team can respond by starting fewer changes until reviewers have room. This is backpressure. In practice, a large change takes more review time than a small one, and a repair may need another review and another CI run. When choosing what to start, the team can use capacity for work that can finish while a higher-priority item waits for a scheduled check, provided it can still meet that appointment. The constraint can also lie beyond review. Uber’s practitioners describe being able to build more product variants than they can feasibly test in experiments.

DevOps Research and Assessment (DORA) measures change lead time from commit to production deployment. That excludes time spent clarifying the request before a commit exists. Its change fail rate captures deployments requiring immediate intervention, while deployment rework rate captures unplanned deployments caused by incidents. Uber’s managed-agent accounting uses still other boundaries, such as cost per review, merged PR, or resolved alert. To compare useful deliveries, follow comparable requests through acceptance and include reviewer effort, integration resources, reverts, escaped defects, and recovery. A migration needing five repository PRs may still fulfill just one request. DORA delivery metrics · Uber’s operating account · AI Cost and Performance Engineering

In early 2025, METR randomized AI availability across 246 tasks performed by 16 experienced open-source contributors in familiar repositories. Using the February–June tools tested, primarily Cursor Pro and Claude 3.5 or 3.7 Sonnet, contributors took 19% longer on tasks where AI was allowed than on tasks without it. They believed the tools had saved time. The unit measured was assisted task completion. In a February 2026 update, METR explained why a later experiment was harder to interpret: contributors withheld tasks or declined participation, some tasks remained incomplete, reported quality differed, and developers did other work during agent runs. Comparing factories requires following unfinished assignments too, and recording human labor separately from elapsed time. METR experiment · METR study-design update

Comparing factory boundaries

An automation boundary is the point at which the system hands a result to someone with the means and authority to act on it. Davis Palmie of Factory proposes a narrow starting point for incident response. An agent reads a Sentry alert, gathers traces of the operations involved, and posts a diagnosis in Slack. An engineer can inspect whether the traces support the diagnosis before committing anyone to a repair. That makes investigation a useful assignment even while implementation and release remain with the service team. Palmie’s signal-to-deployment account

The 2025–2026 Factory, Warp, and Uber accounts cover different assignments. SDC and Google Rapid provide historical comparisons for two of the same decisions: who owns design, and who can promote a build.
System and starting pointArtifact and assessmentBoundary of delegated authority
Factory · Reyes’s repository requestDroid returned a PR after clarification and project checks.A maintainer could review the change. Release was outside the demonstration.
Warp · public issues and contributionsA public dashboard shows assignments and issue state. Lloyd’s design adds specification, code, and product reviews.Maintainers can intervene at those reviews. The proposed loop leaves detailed release and exception permissions unspecified.
Uber · changes, CI failures, and operational signalsManaged agents perform reviews, repairs, triage, and maintenance. Uber tracks outcomes by assignment alongside quality signals.People review work or receive escalations. The operating report leaves final release authority and ownership of uncertain deployments unspecified.
SDC · customer-site system designCentral factory staff performed detailed design, programming, and tests.System design stayed with customer-site managers. Final release authority is unspecified.
Google Rapid · selected integration revisionA named build advanced through configured tests and deployment stages.Project rules and role permissions governed promotion. Rollout could vary by service risk.

Extending an alert investigation to a checked patch takes write access to a bounded workspace, a way to test the repair, and someone able to review it. Extending automation through release also takes evidence about the combined build, permission to expose that class of change, and an operator able to stop the rollout. Review capacity limits how many requests a team can undertake. One interface review may clear a shared dependency for several requests. Each request counts as delivered when its own recipient gets the promised result. Assigning decision rights

Reference questions

Frequently asked questions

How does a software factory relate to DevOps and CI/CD?

A factory organizes repeated software production. DevOps brings development and operations into a shared working relationship. Continuous integration checks frequently combined changes. Continuous delivery keeps a version releasable on demand, and continuous deployment releases passing changes automatically. A factory can use all of these practices. Continuous integration and delivery

Does a factory require several agents?

No. One agent can take an issue and return a draft for tools to check and a maintainer to review. Add workers when their assignments can proceed independently and the team can afford to check the combination. Comparing multi-agent designs

Can work start with incomplete requirements?

Yes. Scope the assignment to something the receiver can already judge, such as identifying consumers of an interface or comparing two designs. Ask the authorized owner to settle consequential choices before implementation depends on them. Factory Missions separates clarification and plan approval from execution. Missions planning workflow

What do passing tests establish?

A passing test shows that the version tested met the stated expectation under the conditions the test exercised. Its setup matters. A test that puts a saved draft in storage and reopens it checks that the editor can read the draft. To assess crash recovery, interrupt an actual editing session and check whether the writer can find and reopen the expected edits. Matching checks to claims

Which changes can run unattended?

An owner can delegate a defined action for an eligible class of changes. The policy should identify the evidence required, where automation stops, and who handles an exception. Permission to open a checked draft can cover a wider class of work than permission to promote it. Release policies in Rapid

Who owns a bad release?

The organization that authorized exposure remains accountable. Its release and service owners need permission to stop rollout, access to restore service, and responsibility for effects already experienced by users. A receiving team must explicitly accept its operating duties. Ownership and authority · Staffing for the required capability

Does more generated code demonstrate value?

More generated code establishes that implementation produced more text. To assess value, follow distinct requests to their agreed finish, observe whether the result serves its intended use, and count waiting, repairs, human effort, and operating failures. One migration spread over several repositories can increase PR volume while delivering a single outcome. Measuring the complete process

Open questions

  1. Which classes of work benefit when automation extends from a checked draft to delivery? Selective assignment can hide difficult tasks, and concurrent agent runs separate elapsed time from human effort. Following comparable requests, including unfinished ones, through acceptance, operating cost, and subsequent repairs would help teams choose where to stop.

  2. When can a factory safely reuse verification after a base revision or dependency changes? Testing every possible combination spends scarce integration capacity, but a missed consumer can break a shared release. Comparing selective checks with full combined runs on the same changes would expose both missed interactions and the cost in builds and waiting.

  3. How can a reviewer improve detection without exhausting the people who act on its findings? Ratings describe visible comments, while omissions can surface much later. Paired reviews on fresh, independently annotated work, coupled with observed human review effort, would help distinguish better detection from a larger volume of plausible comments.

Follow the curated reading path through the speakers and demonstrations behind this entry.

19 min

AI Engineer World's Fair 2026 · 2026

Why Software Factories Fail

Dex Horthy

Cited in this entry

Horthy follows automatically generated patches into the review and testing queue, where more arrivals can increase waiting even when implementation speeds up.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

78 matching talks

TalkSpeakerEventYear
Eno ReyesAI Engineer World's Fair 20262026
Addy OsmaniAI Engineer World's Fair 20262026
Ben SteinAI Engineer World's Fair 20252025
Git push, get an AI API.

Transcript reviewed

Ryan Fox-TylerAI Engineer World's Fair 20242024
Eugene YanAI Engineer World's Fair 20262026
Al HarrisAI Engineer Code 20252025
Alex GavrilescuAI Engineer Code 20252025
How to Kill the Code Review

Transcript reviewed

Ankit JainAI Engineer World's Fair 20262026
Corey J. GallonAI Engineer Code 20252025
Robert BrennanAI Engineer Code 20252025
Agents need more than a chat

Transcript reviewed

Jacob LauritzenAI Engineer Europe 20262026
Ibragim BadertdinovAI Engineer Europe 20262026
Hailong ZhangAI Engineer Summit 20252025
Talha SheikhAI Engineer Europe 20262026
A Genius With Amnesia

Cited in this entry

Victor SavkinAI Engineer World's Fair 20262026
Tariq ShaukatAI Engineer World's Fair 20262026
Itamar FriedmanAI Engineer Code 20252025
Robert BrennanAI Engineer World's Fair 20252025
Anna Marie BenzonAI Engineer World's Fair 20262026
Sumaiya ShrabonyAI Engineer World's Fair 20262026
Christopher Lovejoy, Saul HowardAI Engineer World's Fair 20262026
Raza HabibAI Engineer World's Fair 20242024
Aparna DhinakaranAI Engineer World's Fair 20252025
Zack ProserAI Engineer Europe 20262026
Justin ReockAI Engineer Code 20252025
Louis Knight-WebbAI Engineer Europe 20262026
Build Systems, Not Code

Transcript reviewed

Angie JonesAI Engineer World's Fair 20262026
Davis PalmieAI Engineer World's Fair 20262026
Gautam KorlamAI Engineer NYC 2026 Online Talks2026
The Many Ends of Programming

Transcript reviewed

Ray MyersAI Engineer World's Fair 20252025
AI Platform Engineering

Transcript reviewed

Patrick DeboisAI Engineer World's Fair 20242024
Kyle Penfound, Jeremy Adams - CasañasAI Engineer World's Fair 20252025
Andy Wong, Andrei Bocan, Shane WolfAI Engineer World's Fair 20262026
Ryan CookeAI Engineer World's Fair 20262026
Michal CichraAI Engineer Europe 20262026
Nnenna NdukweAI Engineer World's Fair 20262026
Max Kanat-AlexanderAI Engineer Code 20252025
Luke AlvoeiroAI Engineer Europe 20262026
Sandipan BhaumikAI Engineer Europe 20262026
Killian Carlsen-PhelanAI Engineer World's Fair 20262026
Manish Kapur, Addy OsmaniEvent unknownUnknown
Agents Need Feature Flags

Transcript reviewed

Sachin GuptaAI Engineer World's Fair 20262026
Andrew ThompsonAI Engineer World's Fair 20252025
Mahmoud AbdelwahabAI Engineer NYC 2026 Online Talks2026
Jared JoselowitzAI Engineer World's Fair 20262026
Michele GrantAI Engineer NYC 2026 Online Talks2026
Laurie VossAI Engineer World's Fair 20262026
Ankur DuggalAI Engineer World's Fair 20262026
Sachin GuptaAI Engineer World's Fair 20262026
Nicholas ArcolanoAI Engineer Code 20252025
Field Guide to Fable

Cited in this entry

Thariq ShihiparAI Engineer World's Fair 20262026
Ramana Siddanth EmaniAI Engineer World's Fair 20262026
Suraj GuptaAI Engineer World's Fair 20262026
Allie Howe, Dex Horthy, Geoffrey Huntley, Ian Livingstone, Greg PstruchaAI Engineer World's Fair 20262026
Lou BichardAI Engineer Europe 20262026
Matt PocockAI Engineer Code 2025Unknown
The Highest Loop — swyx

Metadata candidate

Shawn "swyx" WangAI Engineer World's Fair 20262026
Tariq ShaukatEvent unknownUnknown
Allie Howe, Ian Livingstone, Michael Davis, Dick HardtEvent unknownUnknown
Harrison ChaseAI Engineer World's Fair 20252025
Agents Building Agents

Metadata candidate

Alfonso GrazianoAI Engineer World's Fair 20262026
Filip KozeraAI Engineer World's Fair 20252025
Codex and Subagents

Metadata candidate

Vaibhav Srivastav, Katia Gil GuzmanAI Engineer Europe 20262026
Scott WuAI Engineer World's Fair 20252025
Tomas ReimersAI Engineer World's Fair 20252025
Keegan McCallumAI Engineer World's Fair 20252025
Aparna Dhinkaran, Aparna DhinakaranAI Engineer Summit 20252025
Benjamin VerbeekAI Engineer Europe 20262026
David CramerAI Engineer World's Fair 20252025
Elmer Thomas, Maria BermudezAI Engineer World's Fair 20252025
Cole GannawayEvent unknownUnknown
Aparna DhinakaranEvent unknownUnknown
Charlie Guo, Gabriel ChuaAI Engineer World's Fair 20262026
Eric SchwartzAI Engineer Code 2025Unknown
Varun KrovvidiEvent unknownUnknown
Dru KnoxAI Engineer World's Fair 20262026
Jediah KatzEvent unknownUnknown
Stephanie JarmakEvent unknownUnknown

References

Coverage and source review
Processed transcripts
55 processed in full · 5 in the curated path
Automated source review
Passed
Metadata candidates
26 unreviewed; not verified topic membership
Corpus version
c76f4395d1050f81b2bfd27df900224591ef94b026792d1b65896097daa9f77c

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. Ship Production Software in Minutes, Not Months

    https://www.youtube.com/watch?v=iheWKg2Tkrk&t=223s

  2. OpenHands: An Open Platform for AI Software Developers as Generalist Agents

    https://arxiv.org/abs/2407.16741

  3. Factory demo: submitting a task — Ship Production Software in Minutes, Not Months

    https://ai.engineer/talks/iheWKg2Tkrk-ship-production-software-in-minutes-not-months?view=slides#transcript-iheWKg2Tkrk-asset-6

  4. Ship Production Software in Minutes, Not Months

    https://www.youtube.com/watch?v=iheWKg2Tkrk&t=291s

  5. Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar

    https://www.youtube.com/watch?v=7b9tov5Uc5s&t=933

  6. A cloud software factory — Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp

    https://ai.engineer/talks/tUPPVhBBcoM-software-engineering-is-becoming-factory?view=slides#transcript-tUPPVhBBcoM-asset-9

  7. Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp

    https://www.youtube.com/watch?v=tUPPVhBBcoM&t=627

  8. World’s Fair Daily Dispatch — Latent Space

    https://www.latent.space/p/aiewf-daily-dispatch-loops

  9. Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp

    https://www.youtube.com/watch?v=tUPPVhBBcoM&t=657

  10. Agentic SDLC at Uber - Building Blocks for Uber’s Software Factory

    https://www.youtube.com/watch?v=17-YSUHo6Lk&t=619

  11. Agentic SDLC at Uber - Building Blocks for Uber’s Software Factory

    https://www.youtube.com/watch?v=17-YSUHo6Lk&t=842

  12. Software Factories — AI Engineer World’s Fair 2026 program

    https://ai.engineer/worldsfair/schedule

  13. Why Software Factories Fail

    https://www.youtube.com/watch?v=Ib5GBkD555M&t=372

  14. GitHub Copilot: Meet the New Coding Agent

    https://github.blog/news-insights/product-news/github-copilot-meet-the-new-coding-agent/

  15. NATO Software Engineering Conference Report — Mass Produced Software Components

    https://scrummanager.com/files/nato1968e.pdf

  16. Hitachi — History of the Software Business

    https://www-hitachi-co-jp.itdweb.ext.hitachi.co.jp/Div/soft/history.html

  17. System Development Corporation: Defining the Factory Challenge

    https://www.gregorystrachta.com/resources/Touchstones/systemdevelopmen00cusu.pdf

  18. Software Factories: Assembling Applications with Patterns, Models, Frameworks and Tools

    https://s23m.com/oopsla2003/greenfield.pdf

  19. Martin Fowler — Continuous Integration

    https://martinfowler.com/articles/continuousIntegration.html

  20. Continuous Delivery — Patterns

    https://continuousdelivery.com/implementing/patterns/

  21. About devopsdays

    https://devopsdays.org/about

  22. Evaluating Large Language Models Trained on Code

    https://arxiv.org/pdf/2107.03374

  23. Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp

    https://www.youtube.com/watch?v=tUPPVhBBcoM&t=597

  24. The SDLC You Build Yourself — Andy Wong, Andrei Bocan & Shane Wolf, Atlassian

    https://www.youtube.com/watch?v=ojJXgXLUQFc&t=2702

  25. Field Guide to Fable

    https://www.youtube.com/watch?v=9fubhllmsBU&t=759

  26. The Kanban Guide — May 2025

    https://kanbanguides.org/the-kanban-guide/

  27. The Scrum Guide — November 2020

    https://scrumguides.org/scrum-guide.html

  28. The SDLC You Build Yourself — Andy Wong, Andrei Bocan & Shane Wolf, Atlassian

    https://www.youtube.com/watch?v=ojJXgXLUQFc&t=2612

  29. NASA Systems Engineering Handbook — Technical Requirements Definition

    https://www.nasa.gov/reference/4-2-technical-requirements-definition/

  30. Field Guide to Fable

    https://www.youtube.com/watch?v=9fubhllmsBU&t=559

  31. Cucumber — Gherkin Reference

    https://cucumber.io/docs/gherkin/reference/

  32. NASA Systems Engineering Handbook — Product Verification

    https://www.nasa.gov/reference/5-3-product-verification/

  33. Example of the loop — What It Actually Takes to Build a Software Factory — Tereza Tížková, Factory

    https://ai.engineer/talks/vGCJ7diEtrw-what-it-actually-takes-build-software-factory?view=slides#transcript-vGCJ7diEtrw-asset-16

  34. The engineer of the future is the person who is able to choose what is worth doing — Addy Osmani

    https://www.youtube.com/watch?v=n97BCfyFIvw&t=164

  35. The engineer of the future is the person who is able to choose what is worth doing — Addy Osmani

    https://www.youtube.com/watch?v=n97BCfyFIvw&t=834

  36. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=287

  37. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=429

  38. GitHub Docs — Managing a merge queue

    https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue

  39. Site Reliability Engineering — Release Engineering

    https://sre.google/sre-book/release-engineering/

  40. The Site Reliability Workbook — Canarying Releases

    https://sre.google/workbook/canarying-releases/

  41. The Cure for the Vibe Coding Hangover

    https://www.youtube.com/watch?v=JsKTQbT58BY&t=1942

  42. NASA Systems Engineering Handbook — Product Integration

    https://www.nasa.gov/reference/5-2-product-integration/

  43. Automating Large-Scale Refactors with Parallel Agents

    https://www.youtube.com/watch?v=rcsliSIy_YU&t=1192

  44. OpenHands — Automating Massive Refactors with Parallel Agents

    https://www.openhands.dev/blog/automating-massive-refactors-with-parallel-agents

  45. Git Worktree Documentation

    https://git-scm.com/docs/git-worktree

  46. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=571

  47. SLSA v1.2 — Build Provenance

    https://slsa.dev/spec/v1.2/build-provenance

  48. NASA Systems Engineering Handbook — Requirements Management

    https://www.nasa.gov/reference/6-2-requirements-management/

  49. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=503

  50. Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair

    https://people.cs.umass.edu/~brun/pubs/pubs/Smith15fse.pdf

  51. SonarQube + OpenAI: Agentic Development — Killian Carlsen-Phelan, Sonar

    https://www.youtube.com/watch?v=bI8WyG9YB0I&t=103

  52. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=377

  53. How Anthropic Merges 30,000 Code Changes a Day - Manish Kapur, Sonar & Addy Osmani, Anthropic

    https://www.youtube.com/watch?v=AnsXWH2_EBU&t=690

  54. GitHub Docs — About protected branches

    https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches

  55. Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar

    https://www.youtube.com/watch?v=7b9tov5Uc5s&t=90

  56. A Genius With Amnesia

    https://www.youtube.com/watch?v=jVjt-2g8NMY&t=531

  57. Real ROI: Lessons from Enterprises that have already succeeded with LLMs at Scale: Raza Habib

    https://www.youtube.com/watch?v=khcMErjUB8k&t=1030

  58. CI at Scale: Lean, Green, and Fast

    https://arxiv.org/html/2501.03440v2

  59. Site Reliability Engineering — The Evolving SRE Engagement Model

    https://sre.google/sre-book/evolving-sre-engagement-model/

  60. Compensating Transaction pattern

    https://learn.microsoft.com/en-us/azure/architecture/patterns/compensating-transaction

  61. Using RL-based Agent to Detect and Remediate ETL Pipeline Failures

    https://www.youtube.com/watch?v=LrGCT7G_rU8&t=371

  62. PagerDuty: Incidents

    https://support.pagerduty.com/main/docs/incidents

  63. Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar

    https://www.youtube.com/watch?v=7b9tov5Uc5s&t=430

  64. How to Build Quality Gates into Agentic Coding Workflows — Nnenna Ndukwe, Qodo AI

    https://www.youtube.com/watch?v=-AKPRSYccJc&t=740

  65. The Agent Works. Does Anyone Trust It? — Michele Grant, Leading at Scale

    https://www.youtube.com/watch?v=-a2oTS9J67U&t=494

  66. The Site Reliability Workbook — Postmortem Culture: Learning from Failure

    https://sre.google/workbook/postmortem-culture/

  67. From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI

    https://www.youtube.com/watch?v=F0TNSmbo5hE&t=590

  68. uReview: Scalable, Trustworthy GenAI for Code Review at Uber

    https://www.uber.com/us/en/blog/ureview/

  69. Generalization in Adaptive Data Analysis and Holdout Reuse

    https://arxiv.org/html/1506.02629

  70. From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI

    https://www.youtube.com/watch?v=F0TNSmbo5hE&t=2253

  71. Lean Enterprise Institute: Value Stream Mapping

    https://www.lean.org/lexicon-terms/value-stream-mapping/

  72. What Data from 20 Million Pull Requests Reveal About AI Transformation

    https://www.youtube.com/watch?v=WqZq8L-v9pA&t=388

  73. Agentic SDLC at Uber - Building Blocks for Uber’s Software Factory

    https://www.youtube.com/watch?v=17-YSUHo6Lk&t=1047

  74. DORA's software delivery performance metrics

    https://dora.dev/guides/dora-metrics/

  75. Running a Software Factory Efficiently at Uber Scale — Uber Engineering

    https://www.uber.com/us/en/blog/efficient-software-factory/

  76. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

    https://arxiv.org/pdf/2507.09089

  77. METR — We are Changing our Developer Productivity Experiment Design

    https://metr.org/blog/2026-02-24-uplift-update/

  78. The Software Factory: From Bug Report to Production Code — Davis Palmie, Factory

    https://www.youtube.com/watch?v=exiwa9QbQXI&t=433

  79. Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp

    https://www.youtube.com/watch?v=tUPPVhBBcoM&t=264

  80. Running a Software Factory Efficiently at Uber Scale

    https://www.uber.com/us/en/blog/efficient-software-factory/

  81. Factory Missions

    https://docs.factory.ai/missions/overview

  82. Effective harnesses for long-running agents

    https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

  83. Dave Farley — The Deployment Pipeline: Extending the Range of Continuous Integration

    https://continuousdelivery.com/wp-content/uploads/2010/01/The-Deployment-Pipeline-by-Dave-Farley-2007.pdf

  84. GitHub Actions — Deployments and Environments

    https://docs.github.com/en/actions/reference/workflows-and-actions/deployments-and-environments