Contents
  1. Coding agents and automated change
    1. Repository actions and feedback
      1. Where the programmer intervenes
    2. Behavior, scope, and authority
      1. Permission to act
    3. How automated software change developed
  2. Investigating and changing a repository
    1. The working baseline
      1. Concurrent checkouts
    2. Locating behavior and its consumers
    3. Planning around dependencies
    4. Edit formats and their limits
  3. Execution and diagnosis
    1. Reading a command result
    2. Repair and preservation checks
    3. Choosing the next debugging check
      1. Controlling what changes
  4. Human direction and integration
    1. Steering and handing off work
    2. Reviewing the complete change
    3. Which candidate passed?
    4. Checking the integrated result
  5. Evaluating software work
    1. Designing a repository evaluation
    2. Interpreting coding benchmarks
      1. Hidden requirements and exposed solutions
    3. Measuring accepted delivery
  6. Practical reference
    1. Frequently asked questions
      1. How is an agent different from autocomplete?
      2. How much repository context is enough?
      3. Should an agent write tests?
      4. Can passing tests replace review?
      5. When should the agent ask for help?
      6. Does a worktree provide a sandbox?
      7. When does parallel coding help?
      8. Will a benchmark predict results on our repository?
      9. Can an agent merge or deploy?
  7. Check understanding
  8. Open questions
  9. Selected talks
  10. References
  11. Talk library
← All topics

Coding agents

A coding agent is a software system whose language model chooses development tools and uses their results to decide what to do next. It works in a repository, which holds a project’s versioned files and history. With autocomplete, the programmer accepts a suggestion, finds the callers it might affect, and decides which test to run. Cursor Agent can search for those callers, edit the code, and run the test while the programmer inspects the changes in the editor. With GitHub Copilot, the programmer can delegate an issue and review the proposed changes later in a pull request. In either setting, a failed test gives the agent information it can use to choose its next action. Anthropic on how agents choose actions

For an in-person event on this theme, explore Code Summit, our conference on AI coding agents in San Francisco (November 10–12, 2026).

Coding agents and automated change

Repository actions and feedback

In a 2025 demonstration, Anthropic’s Boris Cherny invoked Claude Code from a terminal. The editor displayed a diff, a comparison of two file states. Look at the first argument to the function, then at the decision still open in the terminal.

Cherny’s recorded edit

  • File — index-in-card.js
  • First argument to presentMany — Current users → proposed users || []
  • Permission prompt — “Do you want to make this edit to index-in-card.js?”

The proposed expression passes users through when it is truthy and supplies an empty array otherwise. The decision hinges on what a missing list means. If there are no users, an empty array represents that state. If loading failed, the same fallback could hide the failure. The prompt is still open, so the developer can decline and inspect the caller. Approving the write would leave a separate question for a test: what does the application do when the list is missing?

The model proposes text and tool calls from patterns it learned during training. The harness is the application software around it. In Bill Chen and Brian Fioca’s account it includes “a collection of prompts and tools combined in a core agent loop.” The application checks a requested operation, performs it when permitted, and returns the result. A rejected write and a successful write therefore give the next model call different evidence. A script can also retry an operation, but its author fixes the path through the script. An agent can use the result to choose between reading a file, testing a hypothesis, or asking for a decision. Chen and Fioca on the harness · Harness responsibilities · Agent action selection

Where the programmer intervenes

These 2025–26 workflows illustrate where a programmer can intervene. A single task can involve several handoffs, from inspecting an edit to reviewing a pull request, the proposal to integrate changes into a target branch.
HandoffWhat the agent providesWhat the programmer decides
Editor diffCursor Agent displays edits for inspection and rejection. In Cherny’s Claude Code session, the editor shows the proposed change while the terminal awaits approval.Does the changed expression make sense in this caller?
Terminal sessionOpenAI Codex CLI reads files, edits them, and runs commands in a chosen directory. OpenCode lets developers configure and extend the file and shell tools. Pi starts with a minimal harness and lets developers package extensions, reusable instructions called skills, and prompt templates.Which checkout should the agent use? Which operations and recurring procedures should be available there? Pi creator Mario Zechner on that choice
Delegated taskCognition’s Devin accepts assignments such as a reproducible bug or a bounded migration. At OpenAI’s May 2025 cloud Codex launch, each task ran in its own environment with the repository already loaded.What starting conditions and expected outcome will let me check the work when it returns?
Pull requestGitHub Copilot can submit changes for a delegated issue. Cloud Codex at launch could also propose a pull request.Do the proposed changes and their checks satisfy the issue well enough to merge?

Behavior, scope, and authority

An acceptance criterion describes an outcome someone can observe under stated conditions. In Kiro’s feature specification example, submitting a form with invalid fields should produce field-specific errors. A generic rejection prevents the submission but leaves the user unable to tell which field to correct. A preservation check would submit valid data and confirm that it still succeeds. Together those checks say what should change, what should keep working, and why redesigning the rest of the form is outside the assignment. Behavioral contracts

The kind of change determines the promise. A repair removes a defect. A feature adds an observable capability. A refactoring improves internal structure while preserving observable behavior. For a migration, name the consumers that must work during the transition as well as at its end. Kiro can record requirements, design, and implementation tasks separately. A discovery during implementation may require updating all three. The agent can inspect existing forms to find the project’s field-error component. If the product has no convention for reporting an error that belongs to several fields, the owner may need to decide what the user should see. Independent work can continue in the meantime. Fowler’s definition of refactoring

Permission to act

The task describes the desired software. Authority determines what the agent may do to produce and deliver it. Claude Code distinguishes reading files, modifying them, invoking a shell, and accessing the web. An external service applies its own credential rules. GitHub’s cloud agent can push to a designated branch and submit a draft pull request, while GitHub reserves approving and merging that proposal for humans. Deploying the accepted code requires permission in the destination environment. Claude Code permissions · GitHub’s cloud-agent restrictions

How automated software change developed

One early approach to automatic programming started with what the programmer could state precisely. In 1969, Cordell Green described constructing a program in the course of proving that available operations could produce a specified result. The proof justified the construction relative to its formal description. Even in a restricted form of LISP, finding such a proof could be expensive. Green supplied the proof for a recursive sorting example that the system could not synthesize in reasonable time. Green’s paper

A programmer often knows part of the implementation or can recognize the desired output more easily than formalize the entire job. In 2006, SKETCH let the programmer leave holes in a partial program. The synthesizer searched for fillings that satisfied a behavioral specification within bounds on inputs and execution. Sumit Gulwani’s 2011 spreadsheet system took example input and output strings instead. If two transformations fit every example, the system could ask for the intended output on a cell where they disagreed. The human answer narrowed the search by supplying a missing requirement. SKETCH · Programming by example

For a repair, much of the desired program already exists. GenProg’s 2009 method used a failing test to focus modifications on relevant execution paths, borrowed code structures from the program, then compiled and tested variants. Finding a variant that satisfied both the failing and previously passing tests gave the search a stopping condition. In 2015, Qi and colleagues showed how much depended on that condition. Some checkers accepted incorrect output when the program exited successfully. Limited test coverage also let patches pass after deleting required behavior. GenProg · Analysis of patch plausibility

Code completion drew guidance from regularities in programs people had already written. Hindle and colleagues used frequencies of short token sequences to suggest Java completions in 2012. In 2021 the Codex model generated function implementations, and HumanEval used executable tests to judge candidates. The likelihood of a continuation could guide generation even when its correctness had to be determined by another means. The Codex in that study was a model. The cloud product introduced in 2025 also supplied a repository and execution environment. On the Naturalness of Software · Codex and HumanEval

SWE-bench in 2023 made an issue in an existing repository the assignment. Locating the implementation and understanding its callers became part of solving it. SWE-agent in 2024 investigated how the interface for navigation, editing, and execution affected that work. OpenHands supplied a platform for running such actions. Robert Brennan of All Hands AI described its editor for inspecting and changing files, terminal for running programs, and browser for consulting documentation. These tools matter when the next step depends on a discovery. At a known cursor position, completion can still suffice. For a precisely specified recurring rewrite, a programmed transformation can apply the rule directly. SWE-bench · SWE-agent · Brennan’s introduction

Investigating and changing a repository

The working baseline

Identify the directory the agent will use and the revision it contains. A commit records a saved snapshot. HEAD usually identifies the current branch, or points directly to a commit in a detached checkout. git status adds information a commit identifier cannot supply. It distinguishes changes staged in the index for the next commit, further edits in working files, and untracked files. Ignored files are normally absent from the listing. Inspect existing modifications before assigning credit for an edit or reverting an unsuccessful attempt. Git glossary · Git status

Read the project instructions that apply to the files. Under the AGENTS.md convention, the closest instruction file can specialize a subproject’s commands. Confirm the package, build, and test commands against project and continuous integration configuration. Then check what is actually installed. With pnpm, --frozen-lockfile makes installation fail when the dependency record is missing or inconsistent with the manifest. By contrast, --lockfile-only can update the record without installing the packages. Two checkouts with identical lockfiles can differ in their ability to start the application. pnpm install

Separate files can lead to the same port

Example

The boundary that separates edits ends before the servers bind their sockets.

Repository history
Checkout AOwn working files and index.
Checkout BOwn working files and index.
Server A
Server B
Host address and portShared host networking in this example.
Two checkouts supply different working files to two processes. In this example the first server binds the host address and port. The second attempts the same exclusive bind and encounters a conflict.
Read the diagram as text
  • Repository history.
  • Checkout A. Own working files and index.
  • Checkout B. Own working files and index.
  • Server A.
  • Server B.
  • Host address and port. Shared host networking in this example.
  • Repository history → Checkout A: Supplies A’s starting snapshot.
  • Repository history → Checkout B: Supplies B’s starting snapshot.
  • Checkout A → Server A: Loads A’s files.
  • Checkout B → Server B: Loads B’s files.
  • Server A → Host address and port: Holds exclusive bind.
  • Server B → Host address and port: Second bind fails.

A test suite groups checks for execution together. Run a relevant portion before editing when feasible. Save the input, outcome, and any prerequisite that prevented execution. If a database failed to start, the baseline tells you about setup but leaves the application’s response unknown. If the test reached the application and observed the wrong response, the same input can later distinguish the proposed repair from the starting behavior. Test cases, fixtures, and suites

Concurrent checkouts

A linked Git worktree gives a task its own checked-out files, HEAD, and index while sharing repository history. In Liam Hampton’s demonstration, entering the background agent’s worktree selected the changed application, but starting it exposed a port conflict. Follow the two processes to the same port in the figure. Package caches, credentials, and databases can likewise be shared according to their own configuration. A sandbox imposes separate limits on what a process can read, write, or contact. Git worktrees · Filesystem boundaries · External effects

Example configuration. A fixture reset from one test process can disrupt the other despite separate checkout files. Neutral connectors indicate shared Git history; colored arrows indicate database operations. Worktrees do not provide process or resource isolation, and database sharing is not required.

Locating behavior and its consumers

The model’s context is the information visible to its current call. An agent can have permission to read a million lines while needing only a few to decide where to look next. Aider’s repository map shows selected filenames, symbols, and defining lines such as function signatures. Aider ranks a graph of file dependencies and chooses entries that fit the current conversation’s token budget. A signature can answer which function accepts the relevant fields. Reading its body can answer which fields it rejects. Selecting sufficient information · Revealing detail on demand

Choose the next operation according to what is still unknown.
QuestionOperationEvidence it returns
Where does the error message occur?Search for its literal text.Matching lines, possibly in fixtures, comments, and unrelated code. Ignore rules affect the search.
Which declaration does this name denote here?Go to definition.Locations resolved by the language server, if it can resolve the symbol.
Who depends on this declaration?Find references.Source locations using the symbol. A separately built consumer or dynamic lookup may need another search.
How could execution reach this code?Inspect incoming calls, then read the caller’s conditions.Possible call sites. The selected input and configuration determine which path runs.

Two inputs determine the validator’s behavior

Example

Following calls locates the function. Following configuration reveals which rule it uses.

Submission test
Public handler
Test configuration
Field validator
In this illustrative test, submitted fields reach the validator through the public handler. Configuration supplies the validation mode separately. Inspect both input paths to decide whether the test exercises strict validation.
Read the diagram as text
  • Submission test.
  • Public handler.
  • Test configuration.
  • Field validator.
  • Submission test → Public handler: Invokes with form input.
  • Public handler → Field validator: Passes submitted fields.
  • Test configuration → Field validator: Selects validation mode.

For example, a public submission handler might always call the same validator while a configuration setting selects strict validation. Finding the validator answers where the rule lives. Following the test through the handler and reading its configuration answers whether that test could expose the strict-mode defect. Run the relevant entry point to confirm what happened. Even the set of files edited by a known repair can omit an unchanged consumer that explains the requirement.

When a saved summary disagrees with a changed file, reread the file. When observed behavior disagrees with documentation, preserve the observation and check which behavior the owner intends. A fixture or search result can contain text phrased as a command. It remains input to the investigation and cannot grant permission to execute it. Refreshing context · Keeping instructions and data distinct

Planning around dependencies

Divide a change into pieces with a recognizable outcome, an owner, and a way to check the result. Suppose a catalog lookup that returned a title string will now return a record with a title and an author. The new return shape is a decision both the lookup and its callers depend on. Once the fields are agreed, one developer can update the lookup while others adapt a receipt and a packing slip to print the title. If the callers import generated types, the revised declaration must reach the generator before those callers can be checked against it. Brennan on decomposing refactors

Unit tests can let each caller substitute a catalog that always returns the expected record. That leaves a gap where the changes meet. Print both documents using the real lookup for a sale with a known title. Compare each title line with that title. Comparing the documents only with each other would accept two matching author names. The work can proceed on separate files, but the check needs the changes together. Exercising real integration boundaries · Multi-Agent Systems

Three edits meet at a real sale

Example

The return contract lets three owners proceed. Only their combined code exercises both consumers against the new producer.

Agreed title and author fields
Catalog lookupProduce the record for the requested book.
Receipt generatorPrint the purchased title.
Packing slip generatorPrint the purchased title.
Known-sale checkRun both generators with the real lookup.
In this illustrative change, a catalog lookup returns a record with a title and author. Both document generators depend on its agreed shape. A combined check prints both documents from one known sale and compares each title line with the purchased title.
Read the diagram as text
  • Agreed title and author fields.
  • Catalog lookup. Produce the record for the requested book.
  • Receipt generator. Print the purchased title.
  • Packing slip generator. Print the purchased title.
  • Known-sale check. Run both generators with the real lookup.
  • Agreed title and author fields → Catalog lookup: Specifies return value.
  • Agreed title and author fields → Receipt generator: Specifies field to read.
  • Agreed title and author fields → Packing slip generator: Specifies field to read.
  • Catalog lookup → Known-sale check: Provides real lookup.
  • Receipt generator → Known-sale check: Produces receipt line.
  • Packing slip generator → Known-sale check: Produces packing slip line.

A plan earns its cost when investigation can prevent a costly wrong edit. For a known correction at a unique location, an edit and focused test may be sufficient. For an interface change, locating consumers early can save rewriting several dependent patches. Add a newly discovered consumer to the plan and identify the check it requires. Dividing every operation into its own assignment merely adds handoffs if none can be verified independently. Claude Code’s planning guidance

Edit formats and their limits

An edit format determines what the model must generate and what the editor must locate. Robert Brennan’s OpenHands slide lists three strategies. With full-file replacement, even a one-line correction requires the model to reproduce every line it wants to keep. A diff-based patch specifies the lines to add or remove, with surrounding context to locate them. Find-and-replace supplies an existing passage and its replacement. Both localized formats leave the rest of the file in place, provided the editor can identify the intended site. The editor reports whether it applied the change. A terminal check can then exercise the changed code. Brennan’s editor slide

Slide listing filesystem access and file replacement, diff-based, and find-and-replace editing strategies.
Action Space: The Code Editor — Software Development Agents: What Works and What Doesn't · Robert Brennan · Watch at 4:32

Anthropic’s text editor requires a unique exact match for a replacement, including whitespace. A repeated fragment needs more surrounding text to identify it. A Git patch hunk uses context lines around additions and removals. git apply --check tests whether the hunks fit without writing or reserving the files. By default a failed hunk rejects the patch as a whole. With --reject, some edits can already be on disk and the rejected hunks appear in .rej files. Inspect the new state before retrying. Also check the tool’s vocabulary. Aider calls its search-and-replace format diff and offers a separate simplified unified-diff format. Text editor contract · Git apply · Aider edit formats

For repeated structural changes, a codemod transforms parsed source constructs. LibCST’s concrete syntax tree retains Python comments, parentheses, and whitespace, allowing a transformation to preserve those details around a selected node. It can still select a call whose behavior should stay as it was. After any kind of edit, inspect the actual changed regions and inventory staged, unstaged, and untracked files. A one-line change to a default can affect every caller. Updating several callers together can be the smallest coherent patch. LibCST · Making changes reviewable

Execution and diagnosis

Reading a command result

A terminal tool request contains an operation to attempt. Evidence begins with what the application actually launches. Record the command and arguments, working directory, candidate revision and local edits, relevant configuration, and build loaded or produced. If the executable cannot be found, the application code may never run. If a server starts successfully, it can continue running while a separate client checks its behavior. Tool dispatch · Node.js child processes

To check the form’s field-specific errors, first establish which build the server loaded and which server the client contacted. Each operation below answers a different question about that run.
ActionRecorded resultClaim supported
Build the form application.Build exits successfully and produces the bundle.The configured build completed for those inputs.
Serve that bundle.Server prints a listening address and remains active.A client can attempt a connection to that address.
Probe the submission endpoint.Invalid input returns a response identifying the field.The API identified the field for this request.
Submit through the browser.The browser check finds the error beside the invalid field and exits successfully.The exercised page presented the required error in that location.

Standard output and standard error are distinct streams. A line on either can arrive before the check finishes. An exit code or terminating signal describes how the process ended. In Node.js the close event also waits for its standard I/O streams to close. Check the final assertion and retained output before reporting success. For a background job, track the process through termination. Sending a stop signal alone establishes neither that it exited nor that a shell’s descendants stopped. Stopping owned work

Repair and preservation checks

Run the same defect check on the starting revision and the proposed repair under comparable conditions. If it fails before the edit and passes afterward, the comparison shows that the outcome changed. A pass on the edited version alone cannot establish that. Rerun checks for behavior that should remain intact as well. A new failure there can reveal a regression, a break in previously working behavior. In either case, the test oracle is the rule that decides whether the observed result is acceptable. If that rule checks only part of the requirement, a faulty implementation can earn a pass. Matching checks to claims · Executable outcomes

Naman Jain describes a task requiring the sorted, unique elements common to two lists. A solution that omitted sorting passed the tests. For example, take [7, 3, 7] and [3, 7]. The same outputs receive different verdicts depending on which part of the requirement the assertion checks. Jain’s coding evaluation examples
Candidate outputContains exactly one 3 and one 7Equals the required sequence `[3, 7]`
[7, 3]PassFail
[3, 7]PassPass

In automated repair, a patch that satisfies the validation tests is called plausible. It overfits the tests if it passes them while violating the intended behavior on other inputs. In a libtiff repair analyzed by Qi and colleagues, generated patches removed a faulty integer-overflow guard. The suite accepted the change, which reintroduced a vulnerability. Repairing the guard and deleting the protection could both satisfy the observations available to the search. Patch plausibility and correctness

Choose a check that could detect the specific mistake. Type checking can expose an incompatible declaration. A unit test can distinguish two output sequences. Integration tests exercise real callers and dependencies, while a browser test can inspect what appears after a user interaction. Property-based tests search generated inputs for a violation of a stated rule. Differential tests compare implementations on valid inputs and use disagreement to expose a possible defect. Both depend on the choice of inputs, and two implementations can agree on a wrong answer. QuickCheck · Differential testing with Csmith

Inspect tests the agent changes alongside the implementation. Look for expectations derived from the same algorithm, deleted assertions, newly skipped cases, and edits to the mechanism reporting completion. In Nick Nisi’s Case workflow, an agent created the marker file the gate expected without running the tests. Even a hash of output would need a trustworthy connection to the actual run and candidate. Nisi’s account

Choosing the next debugging check

Preserve the failing input, revision, setup, and result. Then look for a boundary where two explanations predict different observations. At Zed, a repository editing task failed. Inspecting what the model received from a search for fn run exposed a problem upstream of the edit. The search returned a match without enough of the surrounding construct. Zed used Tree-sitter, a code parser, to expand the match to its syntax boundaries. The team could check the expansion on a fixed source file without calling a model. Changing the editing prompt alone would have left the missing source context unexplained. Nathan Sobo’s investigation

For a stale display, inspect the failing interaction before reloading the page. If no request went out, check the code that should have triggered it. If the request went out but produced no usable response, check whether it failed or is still pending. Once a response arrives, compare its value with the displayed value for the same item. An old value in the response sends the investigation upstream. A current value beside a stale display points to the code that consumes the response. A reload that restores the display cannot tell you which step went wrong.

Trace a stale value across the response boundary

Example

A captured response can separate a producer problem from a display problem without rerunning the whole interaction.

Inspect failing refresh
Inspect trigger
Inspect request outcomeFind out whether the request is pending or failed.
Compare the same itemRead the captured response and visible value.
Trace producer
Trace display update
For an interaction that should refresh a value, first determine whether the request ran. If it returned a response, match the item to the displayed item before comparing values. A missing response requires investigating the request before judging either value.
Read the diagram as text
  • Inspect failing refresh.
  • Inspect trigger.
  • Inspect request outcome. Find out whether the request is pending or failed.
  • Compare the same item. Read the captured response and visible value.
  • Trace producer.
  • Trace display update.
  • Inspect failing refresh → Inspect trigger: No request issued.
  • Inspect failing refresh → Inspect request outcome: Issued without usable response.
  • Inspect failing refresh → Compare the same item: Response available.
  • Compare the same item → Trace producer: Response already stale.
  • Compare the same item → Trace display update: Response current, display stale.

Controlling what changes

A successful rerun after several changes rarely identifies the cause. Suppose a date test passes alone on the base revision but fails after a language-setting test on the patch. Run the date test alone on the patch and the two-test sequence on the base. Use a fresh process for each run and hold the rest of the setup fixed. If both new runs pass consistently, only the combination of the patch and that test order fails among the four conditions. If the sequence also fails on the base, investigate the pre-existing dependency on process state before attributing the failure to the patch. pytest on flaky tests · Testing explanations

With a repeatable failure and known good and bad revisions, git bisect narrows the search by selecting intermediate revisions to rebuild and test. Skip a revision that fails to build for an unrelated reason, even if the introducing commit remains uncertain. In a live session, set aside a speculative edit if it obscures the reproduction. When another retry cannot distinguish any remaining explanations, inspect a different boundary or ask for the missing information. Git bisect · Bounding retries

Human direction and integration

Steering and handing off work

Interactive work lets a programmer correct an assumption before the agent builds more code around it. In an Aider session, the programmer wanted tests to simulate terminal input. After Aider changed the tests, the programmer reported that the application’s main function rejected the new input argument. Changing the test alone could never deliver that input to the component that read it. The programmer identified where input and output needed to go, supplied the library signatures, and interrupted an approach that used the wrong interface. Aider switched to PromptSession, and the programmer reported that it worked. Aider’s debugging session

For background work, choose an assignment whose outcome the returning reviewer can recognize. A performance fix should arrive with the affected workload, evidence of where time went, the change, and a rerun checking its effect on that workload. Interrupt the programmer if a choice would change the goal or require more authority. For a discoverable fact such as a library signature, inspect the installed version first. A progress update should say which observation changed the plan and what decision, if any, remains with the owner. Verifying background performance work

A handoff identifies the current goal, checkout, revision, working edits, active processes, completed checks, and outstanding work. If direction changed, record the superseded decision and which checks depended on it. For example, an API test could still establish that the response identifies the invalid field after a change to error placement, while the browser assertion needs updating. On resumption, inspect the files and process state before continuing. A summary can be stale and a stopped conversation can leave both edits and a server behind. Reconstructing context · Handling waits and approvals

Reviewing the complete change

Name the states being compared. git diff A B compares the two commit snapshots. git diff A...B compares their common merge base with B, showing what the branch contributes since divergence. In a working checkout, plain git diff compares working files with the index. git diff --cached compares staged content with a commit, normally HEAD. Check status as well. An untracked helper or unstaged correction might explain why a local run passed while the proposed commit is incomplete. Git diff

Follow the requirement through the entry point, changed code, and unchanged consumers. Inspect configuration, dependencies, generated files, and modifications to the tests. For the libtiff patch, a reviewer could ask where the changed program now prevents the overflow. That question connects a visible deletion to a protection the suite failed to exercise. An AI reviewer can raise such a hypothesis. Ask for its consequence, the source path that permits it, and a check or independently established requirement. Agreement between two models still permits both to share the same mistaken expectation. Google’s code review guidance · Presenting evidence for review

A review also needs someone responsible for answering it. Brennan reported that early OpenHands pull requests appeared under the bot’s identity. Under his team’s rules, the initiating programmer could then approve the request, bypassing the intended second human review. Failed checks could sit unanswered. The team changed attribution so the initiating human owned the proposal and its consequences. Brennan on pull-request ownership

Which candidate passed?

The candidate is the software proposed for acceptance. Tie each result to the source revision, relevant working files, dependency and configuration state, and generated or compiled artifact the check actually loaded. A browser test pointed at an old bundle can pass after an edit to the source. A completion report should identify the command, the candidate it exercised, the behavior it checked, and its outcome. Identifying what was tested

In the figure, a title assertion passes on A. Editing the receipt creates B. The expected title has stayed the same, but the code producing the line has changed. Exercise the receipt built from B and record its own result. Evidence about an unaffected property can carry forward if you can identify the relevant inputs and show they still apply. Review approval can also expire. GitHub branch protections can dismiss an approval when a new push or a changed base alters the reviewed diff. Protected branches

One assertion, two executions

Example

Editing the receipt opens a gap in the evidence even when the expected title stays the same.

1 / 3 · Print the receipt on A

The title assertion passes on A.

New this stepCandidate A
New this stepReceipt title assertionCompare the title printed for a known sale with the purchased title.
New this stepPass on A
A receipt title passes its assertion on candidate A. Candidate B changes the receipt code. A’s historical result persists while B awaits a check. Running the assertion on B supplies a separate result for B.
Read the diagram as text
  • Candidate A.
  • Receipt title assertion. Compare the title printed for a known sale with the purchased title.
  • Pass on A.
  • Candidate B. Receipt implementation edited.
  • Title unchecked on B.
  • Pass on B.
  • Candidate A → Pass on A: Version exercised.
  • Receipt title assertion → Pass on A: First execution.
  • Candidate A → Candidate B: Receipt edit creates.
  • Candidate B → Title unchecked on B: Needs new result.
  • Candidate B → Pass on B: Version exercised.
  • Receipt title assertion → Pass on B: Fresh execution.
  1. Print the receipt on A. The title assertion passes on A. Active: Candidate A, Receipt title assertion, Pass on A. New: Candidate A, Receipt title assertion, Pass on A.
  2. Edit the receipt. The earlier observation still describes A. B has different receipt code. Active: Candidate A, Receipt title assertion, Pass on A, Candidate B, Title unchecked on B. New: Candidate B, Title unchecked on B.
  3. Print the receipt on B. The check now records an outcome for B as well. Active: Candidate A, Receipt title assertion, Pass on A, Candidate B, Pass on B. New: Pass on B.

Distinguish a pass and a failure, both observed outcomes, from a skipped check or an unavailable assessment. A runner can intentionally omit a case. A service that failed to start can prevent the runner from judging it at all. Some GitHub status rules accept skipped or neutral statuses. Before telling a reviewer the behavior passed, inspect what ran on the final candidate. Completion checks

Checking the integrated result

Merging combines versioned changes. A textual conflict leaves edits Git cannot reconcile automatically. Inspect the common base and both contributors’ changes to preserve their respective intentions. A semantic conflict can survive a clean text merge. In a reported Microsoft Edge merge, Chromium had renamed a member from IsIncognito to GetIncognito. Downstream code still called IsIncognito. Git could combine those files, but the build failed when the compiler tried to resolve the old name. Git merge · Microsoft’s Edge merge study

If the target branch has changed, merge the current target and check the resulting code. Inspect and exercise the combined result. A runtime conflict can be subtler than a missing member. If reading a saved value can silently fall back to a current default, first save one value, then change the default. Exercise the combined code and assert the saved value. With equal values, the assertion would pass along either path.

A clean merge with an unresolved member

The name supplied by the upstream declaration differs from the name the downstream caller needs.

Upstream declarationGetIncognito
Downstream callerIsIncognito
Combined sourceGit reconciles the files.
Member lookup failsThe compiler cannot resolve the downstream call.
In the reported Edge merge, Chromium supplied the renamed member GetIncognito while the downstream call still requested IsIncognito. The files merged without a textual conflict. The combined build failed to resolve the old name.
Read the diagram as text
  • Upstream declaration. GetIncognito
  • Downstream caller. IsIncognito
  • Combined source. Git reconciles the files.
  • Member lookup fails. The compiler cannot resolve the downstream call.
  • Upstream declaration → Combined source: Defines GetIncognito.
  • Downstream caller → Combined source: Calls IsIncognito.
  • Combined source → Member lookup fails: Build looks up IsIncognito.

Continuous integration (CI) runs automated checks on shared changes. GitHub’s merge queue constructs a temporary candidate from the proposal, current target, and preceding queued work. Its commit identifier differs from the pull-request head. Required GitHub Actions workflows need to handle merge_group to report checks on that combination. A submission opens a proposal, a review records judgment, a merge changes the target history, and deployment changes what runs in an environment. Verify and authorize each transition separately. Reverting a commit afterward can leave database writes or other external effects intact. GitHub merge queues · Restoring software after effects · Software Factories

Evaluating software work

Designing a repository evaluation

Verification asks whether a particular change satisfies a requirement. Evaluation measures how a coding system performs across the work you intend to delegate. Each case specifies an assignment and its starting state. Each trial is one attempt at that case. The starting information matters in maintenance work. Handing the agent the commit that introduced a defect narrows its search. Asking it to find defects first introduces a selection effect: if it finds mostly simple bugs, its repair rate describes that easier subset. Every bug it missed is absent from the repair count. The SM100 maintenance evaluation · Cases and trials · Representative tasks

A SWE-bench repository issue separates materials according to who uses them. Supplying the reference fix to the solver would change an issue-resolution task into a different assignment. SWE-bench dataset structure
RoleMaterialsUse in a comparison
Solver inputsIssue, repository at its base commit, installed environmentGive each system the same starting problem and allowed information.
Trial outputsCandidate patch and execution logsRetain what each system produced, including unsuccessful attempts.
Assessment materialsReference fix, test patch, designated repair and preservation checksKeep the reference fix out of solving. Use the same grading procedure on each submission.

Define success according to the assignment. Check a repair against the failing case and preserved behavior, a feature through its public entry point, a refactoring against its preservation promise, and integration against the named target. Specify the model and harness, allowed tools and lookup, context policy, human help, time and resource limits, number of attempts, stopping and selection rules, and grader version. Retain enough evidence to distinguish an incorrect submission from a broken environment, a grader mistake, and a case whose cause is still unknown. Ibragim Badertdinov on executable issue tasks · SWE-bench evaluation guide

Two controls solve different problems. Resetting the environment prevents one attempt from inheriting another’s files or processes. Reserving untouched cases prevents earlier grades from shaping the system selected for final evaluation. Even an aggregate score can guide tuning. Keep discovered failures as regression cases, freeze the configuration, then evaluate on fresh representative tasks. Match resource limits between systems. Anthropic found that extra memory could prevent an infrastructure failure or enable a different solution strategy even with the model, tasks, and harness fixed. Infrastructure noise · Paired comparisons

Interpreting coding benchmarks

Read a score by reconstructing one successful instance.
BenchmarkGivenSubmittedWhat passes
HumanEvalA function promptA generated implementationThe function satisfies the executable tests.
Original SWE-benchAn issue and a repository revisionA repository patchThe patch applies. Designated fail-to-pass tests that exposed the problem now pass, and designated pass-to-pass tests protecting existing behavior keep passing.

Resolved rate counts the fraction of instances meeting the repository benchmark’s grading rule. Identify the dataset revision, split, and grader implementation. For example, the SWE-bench grader’s pass-and-fail mode treats a skipped fail-to-pass test as a failure yet can count a skipped pass-to-pass test as maintained. A different mode has different rules for missing results. Knowing the label alone does not tell you which code ran. Grader implementation

Pass@k measures coverage by at least one passing candidate among k samples. If only the fourth of ten patches passes, the ten contain a success. A system that returns the first patch still hands the programmer a failing change. To interpret the result for deployment, ask how much sampling was allowed, which checks the agent could consult, and which candidate its selection policy actually returned. Attempts and selection

Hidden requirements and exposed solutions

A grader can also reject a reasonable solution. An original SWE-bench issue said that TfidfVectorizer ignored its copy argument. The resolving pull-request discussion later specified a deprecation warning, and the hidden test required its exact text. A solver reading the issue lacked that decision. In August 2024, SWE-bench Verified selected 500 tasks after developers screened issues and tests for problems including hidden requirements. Introducing SWE-bench Verified

In February 2026, OpenAI reported further problems in Verified. Investigators concentrated on the 27.6% of tasks models often failed. They judged at least 59.4% of that selected subset capable of rejecting functionally correct solutions. That percentage describes the audited subset. On some tasks, models also reproduced distinctive problem text or reference fixes, which OpenAI interpreted as evidence of prior exposure. It stopped reporting Verified for frontier launches. Prior exposure during training and retrieving a public fix during a trial are different routes to a misleading success. DeepSWE uses newly authored tasks from contributors familiar with the repositories to reduce reliance on issues with already published solutions. OpenAI’s reassessment · DeepSWE task design

Measuring accepted delivery

A patch may arrive in minutes and wait hours for someone who understands the affected service. To measure useful delivery, define a common acceptance standard and follow comparable assignments through that point. Count abandoned work, review, corrections, integration effort, and operating expense. Track later repairs where they are observable. Separate elapsed time from human attention. A programmer who works on another issue while an agent runs incurs a wait on the first issue’s clock without necessarily spending that interval supervising it. A teammate who reviews the patch contributes time even if the author has moved on. Comparison baselines · AI Cost and Performance Engineering

METR randomized whether 16 experienced open-source developers could use AI on 246 predefined tasks in repositories they knew well. Allowing the early-2025 tools increased measured completion time by 19% relative to working without them. Participants estimated afterward that the tools had saved time. The discrepancy matters because a survey and a clock can give opposite answers about the very same assignments. The study gives a result for those developers, tools, and tasks. METR’s randomized study

By February 2026, METR reported that some developers declined participation or withheld tasks they were unwilling to do without AI. Completion and delivered work differed, and concurrent assignments made task time harder to interpret. METR judged the later data unreliable for estimating the current effect. In a local comparison, record which assignments enter the study, who already knows the repository, whether the work is supervised, who reviews it, and what counts as accepted. More pull requests can reflect smaller task divisions or more review work without settling whether useful delivery improved. METR’s experiment redesign

Practical reference

Frequently asked questions

How is an agent different from autocomplete?

Autocomplete suggests text at an editing position. The programmer decides what to do with it. A coding agent can choose to inspect a caller, request an edit, or run a test and use the returned result to choose its next operation. The programmer can intervene at any decision the interface exposes.

How much repository context is enough?

Gather enough to trace the relevant entry point, affected consumers, and a check that would expose a wrong change. Use a repository map to choose the next file. Read its implementation or configuration when the answer depends on details omitted from the map.

Should an agent write tests?

Yes. Derive the expected result from the requirement or an independently checked example. Ask whether a plausible faulty implementation would fail the assertion. A test that computes its expectation with the same faulty logic can pass consistently.

Can passing tests replace review?

A passing suite supports the properties it actually exercised. Review can reveal an omitted caller, a deleted guard, or an assertion weakened by the patch itself. Inspect those changes before accepting the result.

When should the agent ask for help?

Inspect project commands, library signatures, and existing conventions in the checkout first. Ask when the remaining choice changes what users receive, requires information only the owner has, or exceeds the assigned authority. A recurring procedure can be recorded for later sessions. Agent Skills

Does a worktree provide a sandbox?

A worktree supplies a separate checkout and staging area. A process launched there can still bind the host’s ports or contact shared services with available credentials. Containment depends on configured filesystem, network, and execution controls.

When does parallel coding help?

Parallelize when each assignment has a clear owner, its dependencies are settled, and its result can be inspected. Where the changes meet, run the real consumers against the combined candidate. Multi-Agent Systems

Will a benchmark predict results on our repository?

Transfer depends on how closely the tasks, tools, available information, assistance, and acceptance checks match your workflow. For a local estimate, fix the system configuration and assess representative assignments held out from development.

Can an agent merge or deploy?

Only when the task owner delegates the specific operation and the destination permits it. GitHub’s cloud agent can submit its work on a designated branch but cannot approve or merge its pull request. A separate destination grant governs deployment. Software Factories

Open questions

  1. How can an agent tell when it needs a product decision before it writes both code and tests around the same assumption? Agreement between implementation and test can conceal the mistake. Progress would mean identifying a small distinguishing example that an owner can resolve, while allowing different implementations of the agreed behavior.

  2. Which results remain valid after a target branch advances? An unchanged caller can depend on a renamed declaration, a configuration value, or a generated artifact. A useful method would explain why each retained check still applies and compare its predictions with checks actually run on the combined candidate.

  3. When does unattended work reduce total human effort through acceptance and maintenance? Task selection changes with adoption, waiting overlaps other work, and review can move to another person. Comparable assignments with separate measures of elapsed time, active attention, rejected work, and later repair would reveal where the savings or costs accrue.

Follow the curated reading path through the speakers and demonstrations behind this entry.

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

591 matching talks

TalkSpeakerEventYear
Morgante PellAI Engineer World's Fair 20242024
Aakanksha ChowdheryAI Engineer World's Fair 20252025
Corey J. GallonAI Engineer Code 20252025
Maxime Rivest, Isaac MillerAI Engineer World's Fair 20262026
Mark MoyouAI Engineer World's Fair 20242024
Julián Duque, Anush DSouzaAI Engineer World's Fair 20252025
Danilo CamposAI Engineer Europe 20262026
Scaling to Long Horizons

Transcript reviewed

Ross Taylor, Chengxi TaylorAI Engineer World's Fair 20262026
Aditya BhargavaAI Engineer World's Fair 20262026
Nick NisiAI Engineer Europe 20262026
Mason EggerAI Engineer World's Fair 20252025
Frank LiuAI Engineer World's Fair 20252025
Future-Proof Coding Agents

Cited in this entry

Bill Chen, Brian FiocaAI Engineer Code 20252025
Jesse HuAI Engineer Code 20252025
Yogendra MirajeAI Engineer World's Fair 20252025
Gaurav MishraAI Engineer World's Fair 20262026
Steven WillmottAI Engineer Europe 20262026
How to Kill the Code Review

Cited in this entry

Ankit JainAI Engineer World's Fair 20262026
Darius EmraniAI Engineer World's Fair 20252025
Fouad MatinAI Engineer World's Fair 20252025
Harshil AgrawalAI Engineer Europe 20262026
Security Firewall for Agents

Transcript reviewed

Ryan DahlAI Engineer World's Fair 20262026
Moritz JohnerAI Engineer World's Fair 20262026
Eric AllamAI Engineer World's Fair 20252025
Samuel ColvinAI Engineer Code 20252025
Luke AlvoeiroAI Engineer Europe 20262026
Ramana Siddanth EmaniAI Engineer World's Fair 20262026
Ara KhanAI Engineer Europe 20262026
Pierluca D'OroAI Engineer World's Fair 20262026
Naman JainAI Engineer Code 20252025
Jun Yu TanAI Engineer World's Fair 20252025
Angel Ortmann LeeAI Engineer World's Fair 20262026
What's next after RLHF?

Transcript reviewed

Diogo AlmeidaAI Engineer World's Fair 20262026
Sam MorrowAI Engineer Europe 20262026
Alex GavrilescuAI Engineer Code 20252025
Hailong ZhangAI Engineer Summit 20252025
Talha SheikhAI Engineer Europe 20262026
David GomesAI Engineer Europe 20262026
Liam HamptonAI Engineer Europe 20262026
Kevin HouAI Engineer World's Fair 20242024
Nupur SharmaAI Engineer Europe 20262026
Eno ReyesAI Engineer World's Fair 20242024
Robert BrennanAI Engineer Code 20252025
How Claude Code Works

Transcript reviewed

Jared ZoneraichAI Engineer Code 20252025
Vinoth GovindarajanAI Engineer World's Fair 20262026
Ash Prabaker, Andrew WilsonAI Engineer Europe 20262026
Josh AlbrechtAI Engineer World's Fair 20252025
Tomas ReimersAI Engineer World's Fair 20252025
Hugo Santos, Madison FaulknerAI Engineer Europe 20262026
2025 in LLMs so far

Transcript reviewed

Simon WillisonAI Engineer World's Fair 20252025
Rishi DesaiAI Engineer World's Fair 20262026
Ali KhialAI Engineer World's Fair 20262026
Sachin GuptaAI Engineer World's Fair 20262026
Amazon AGI

Cited in this entry

Amazon AGI, Aditya KhandelwalAI Engineer World's Fair 20262026
Rafal Wilinski, Vitor BaloccoAI Engineer World's Fair 20252025
Building Self-Coding Agents

Cited in this entry

Colin FlahertyAI Engineer Summit 20252025
Al HarrisAI Engineer Code 20252025
Sarthak AggarwalAI Engineer World's Fair 20262026
Eashan SinhaAI Engineer World's Fair 20252025
Davis PalmieAI Engineer World's Fair 20262026
Rajkumar SakthivelAI Engineer World's Fair 20262026
Daksh GuptaEvent unknownUnknown
Agents need more than a chat

Cited in this entry

Jacob LauritzenAI Engineer Europe 20262026
Daksh GuptaAI Engineer World's Fair 20262026
Matt BrockmanAI Engineer World's Fair 20262026
Xander SongEvent unknownUnknown
Laurie VossAI Engineer World's Fair 20262026
Sarah ChiengAI Engineer Europe 20262026
Ido SalomonAI Engineer Europe 20262026
May WalterAI Engineer World's Fair 20262026
Vibes won't cut it

Transcript reviewed

Chris KellyAI Engineer World's Fair 20252025
Taylor Luttrell-Williams, Killian Carlsen-PhelanEvent unknownUnknown
Maggie AppletonAI Engineer Europe 20262026
A Genius With Amnesia

Transcript reviewed

Victor SavkinAI Engineer World's Fair 20262026
Ryan Lopopolo, Vibhu SapraAI Engineer Europe 20262026
Rafal WilinskiAI Engineer World's Fair 20262026
James ShiAI Engineer World's Fair 20262026
Yegor Denisov-BlanchAI Engineer World's Fair 20252025
Gergely Orosz, swyxAI Engineer Europe 20262026
Lei ZhangAI Engineer Code 20252025
Matthias LuebkenAI Engineer Europe 20262026
Matt PocockAI Engineer Europe 20262026
Beyang LiuAI Engineer Code 20252025
Denys LinkovAI Engineer World's Fair 20262026
Jon Peck, Christopher HarrisonAI Engineer World's Fair 20252025
Devendra Chaplot, Devendra Singh ChaplotAI Engineer World's Fair 20242024
Max Kanat-AlexanderAI Engineer Code 20252025
Ahmad AwaisAI Engineer Code 20252025
KitzeAI Engineer World's Fair 20252025
Nik PashAI Engineer Code 20252025
Ian ButlerAI Engineer World's Fair 20252025
Mahmoud AbdelwahabAI Engineer Code 20252025
Recursive Coding Agents

Metadata candidate

Raymond WeitekampAI Engineer World's Fair 20262026
Marc KlingenAI Engineer Europe 20262026
Beyang LiuAI Engineer World's Fair 20252025
Harald KirschnerAI Engineer World's Fair 20252025
Harald KirschnerAI Engineer World's Fair 20252025
Itamar FriedmanAI Engineer World's Fair 20252025
Daniel SzokeAI Engineer Europe 20262026
Rustin BanksAI Engineer World's Fair 20252025
Ben BurtenshawAI Engineer Europe 20262026
Patrick DeboisAI Engineer World's Fair 20262026
Robert BrennanAI Engineer World's Fair 20262026
Jakub HojsanAI Engineer Code 2025Unknown
Eyal BlumAI Engineer World's Fair 20262026
Kamalakannan NandagopalEvent unknownUnknown
Dan AdlerAI Engineer World's Fair 20262026
Nnenna NdukweAI Engineer World's Fair 20262026
Sheilah KiruiAI Engineer World's Fair 20262026
Ido SalomonAI Engineer World's Fair 20262026
Karan VaidyaEvent unknownUnknown
Justin ReockEvent unknownUnknown
Kenton VardaAI Engineer World's Fair 20262026
Ishan AnandAI Engineer World's Fair 20242024
Nuno CamposAI Engineer Europe 20262026
Felippe Felisola Caso, Francesco GallettaEvent unknownUnknown
Jess WangEvent unknownUnknown
Andrew GarvinAI Engineer World's Fair 20262026
Diego CarpinteroAI Engineer Europe 20262026
2026: The Year the IDE Died

Metadata candidate

Steve Yegge, Gene KimAI Engineer Code 20252025
Harrison ChaseAI Engineer World's Fair 20252025
A Song of Types and Agents

Metadata candidate

Roberto StagiAI Engineer World's Fair 20262026
Will Hang, Cathy ZhouAI Engineer Code 20252025
Ezra Tanzer, Dan ArpinoAI Engineer World's Fair 20262026
Brendan O'LearyAI Engineer Europe 20262026
Steve YeggeAI Engineer World's Fair 20262026
Kevin HouAI Engineer Summit 20252025
Agents Building Agents

Metadata candidate

Alfonso GrazianoAI Engineer World's Fair 20262026
Shawn "swyx" WangAI Engineer Europe 20262026
AGI: The Path Forward

Metadata candidate

Eiso Kant, Jason WarnerAI Engineer Code 20252025
Rajat ShahAI Engineer World's Fair 20262026
Christopher ChedeauAI Engineer World's Fair 20252025
Dax RaadAI Engineer Code 20252025
Boris Bogatin, Toufic BoubezAI Engineer Code 20252025
Olivier Leplus, Yohan LasorsaAI Engineer Europe 20262026
AI Engineering 101

Metadata candidate

Noah HeinAI Engineer Summit 20232023
swyxAI Engineer World's Fair 20242024
AI SDK v6

Metadata candidate

Nico AlbaneseAI Engineer Europe 20262026
Brendan RappazzoAI Engineer World's Fair 20262026
Justin SmithAI Engineer World's Fair 20262026
Anthropic for VPs of AI

Metadata candidate

Alexander Bricken, Joe BayleyAI Engineer Summit 20252025
Michal CichraAI Engineer Europe 20262026
Rajiv ChandegraAI Engineer World's Fair 20262026
Paul Klein IVAI Engineer World's Fair 20262026
Will BrykAI Engineer World's Fair 20252025
Michael HablichAI Engineer Europe 20262026
Bruno Passos, Beyang LiuAI Engineer Summit 20252025
Building AI For All

Metadata candidate

Amjad Masad, Michele CatastaAI Engineer Summit 20232023
Bennet FennerAI Engineer Europe 20262026
Thor Schaeff, Philipp SchmidAI Engineer Europe 20262026
Building Cursor Composer

Metadata candidate

Lee RobinsonAI Engineer Code 20252025
Peter WielanderAI Engineer Code 20252025
Building security around ML

Metadata candidate

Dr. Andrew DavisAI Engineer World's Fair 20242024
Eric ZakariassonAI Engineer Europe 20262026
Prasenjit SarkarAI Engineer Europe 20262026
Cat Wu, Thariq Shihipar, Simon WillisonAI Engineer World's Fair 20262026
Dylan PatelAI Engineer World's Fair 20242024
Containing Agent Chaos

Metadata candidate

Solomon HykesAI Engineer World's Fair 20252025
Dex HorthyAI Engineer Code 20252025
Context Is the New Code

Metadata candidate

Patrick DeboisAI Engineer Europe 20262026
Copilots Everywhere

Metadata candidate

Thomas Dohmke, Eugene YanAI Engineer World's Fair 20242024
Karina NguyenAI Engineer Summit 20252025
Vincent KocAI Engineer Europe 20262026
Develop at Idea Velocity

Metadata candidate

Jeffrey Lee-ChanAI Engineer World's Fair 20262026
Tomas ReimersAI Engineer World's Fair 20252025
Philipp SchmidAI Engineer World's Fair 20262026
Barry ZhangAI Engineer Summit 20252025
Joseph Wang, SidAI Engineer World's Fair 20262026
Garry TanAI Engineer World's Fair 20262026
Sam BhagwatAI Engineer World's Fair 20262026
Theo BrowneAI Engineer World's Fair 20262026
Danny Gollapalli, Ben Hylak, Zubin KotichaAI Engineer Europe 20262026
Daniel Kim, Daria SobolevaAI Engineer World's Fair 20252025
Chris NoringAI Engineer Europe 20262026
Frontier Feud

Metadata candidate

Barr Yaron, Mihir, John, Tina, Shresta, Paige, Colin, Petra, StevenAI Engineer Summit 20252025
Ilan BigioAI Engineer Summit 20252025
Cassidy HardinAI Engineer Europe 20262026
Dave BurnisonAI Engineer World's Fair 20242024
John PhamAI Engineer World's Fair 20252025
Anirban ChatterjeeAI Engineer World's Fair 20262026
Why Software Factories Fail

Metadata candidate

Dex HorthyAI Engineer World's Fair 20262026
Donald HruskaAI Engineer World's Fair 20252025
Brian ScanlanAI Engineer Europe 20262026
Benjamin VerbeekAI Engineer Europe 20262026
Beth GlenfieldAI Engineer World's Fair 20252025
Kyle Jaejun LeeAI Engineer World's Fair 20262026
Tariq ShaukatAI Engineer World's Fair 20262026
Jake NationsAI Engineer Code 20252025
Yu SuAI Engineer World's Fair 20262026
Chip HuyenAI Engineer Summit 20252025
Lawrence JonesAI Engineer Europe 20262026
Justin ReockAI Engineer Code 20252025
Juan PeredoAI Engineer Summit 20252025
Michael RichmanAI Engineer Europe 20262026
Manuel OdendahlAI Engineer World's Fair 20242024
Kyle MisteleAI Engineer World's Fair 20262026
Eno ReyesAI Engineer Code 20252025
David CramerAI Engineer World's Fair 20252025
Theodora ChuAI Engineer World's Fair 20252025
Mentoring the Machine

Metadata candidate

Eric HouAI Engineer World's Fair 20252025
Peter Werry, BrandonAI Engineer Europe 20262026
Minimax M2

Metadata candidate

Olive SongAI Engineer Code 20252025
Move Fast Break Nothing

Metadata candidate

Dedy KredoAI Engineer Summit 20232023
Martin Harrysson, Natasha ManiarAI Engineer Code 20252025
Arjun SinghAI Engineer World's Fair 20262026
Rémi LoufAI Engineer World's Fair 20242024
No More Slop – swyx

Metadata candidate

Shawn "swyx" WangAI Engineer Code 20252025
On AI and Knowledge

Metadata candidate

Pablo CastroAI Engineer World's Fair 20262026
Saoud RizwanAI Engineer World's Fair 20262026
Ryan MartenAI Engineer World's Fair 20252025
Philip KielyAI Engineer World's Fair 20252025
Christopher HarrisonAI Engineer World's Fair 20252025
Proactive Agents

Metadata candidate

Kath KorevecAI Engineer Code 20252025
Chris ParsonsAI Engineer Europe 20262026
Harald Kirschner, Christopher HarrisonAI Engineer World's Fair 20252025
Jon PeckAI Engineer World's Fair 20252025
Idan GazitAI Engineer World's Fair 20262026
Recursive Model Improvement

Metadata candidate

Lee RobinsonAI Engineer World's Fair 20262026
Connor AdamsAI Engineer Europe 20262026
Benoit SchillingsAI Engineer World's Fair 20262026
Respect The Process

Metadata candidate

Andrew DumitAI Engineer World's Fair 20262026
Patrick DeboisAI Engineer Summit 20252025
Shashi JagtapAI Engineer World's Fair 20262026
Michael YuanAI Engineer World's Fair 20252025
Onur SolmazAI Engineer Europe 20262026
Gunjan PatelAI Engineer World's Fair 20242024
Eno ReyesAI Engineer World's Fair 20252025
Skills are the New SDKs

Metadata candidate

Elvin AghammadzadaAI Engineer World's Fair 20262026
Louis Knight-WebbAI Engineer Europe 20262026
Matt PocockAI Engineer Europe 20262026
Gus Martins, Ian BallantyneAI Engineer Europe 20262026
The New Code

Metadata candidate

Sean GroveAI Engineer World's Fair 20252025
Sarah GuoAI Engineer World's Fair 20252025
Brandon WaselnukAI Engineer Europe 20262026
Rob CheungAI Engineer World's Fair 20242024
Vikash Agrawal, LindaAI Engineer World's Fair 20252025
Brendan O'DonoghueAI Engineer Europe 20262026
Michele CatastaAI Engineer Code 20252025
Patrick DeboisAI Engineer World's Fair 20252025
Christopher Harrison, John PeckAI Engineer World's Fair 20252025
Jack CableAI Engineer World's Fair 20262026
Quinn SlackAI Engineer World's Fair 20242024
Ado KukicAI Engineer Summit 20232023
Natalie MeurerAI Engineer World's Fair 20262026
Travis Bartley, Myungjong Kim, Byungjoong, JaehanAI Engineer World's Fair 20252025
Sam FertigAI Engineer World's Fair 20252025
Armin Ronacher, Cristina Poncela CubeiroAI Engineer Europe 20262026
Allie Howe, Dex Horthy, Geoffrey Huntley, Ian Livingstone, Greg PstruchaAI Engineer World's Fair 20262026
Itamar FriedmanAI Engineer World's Fair 20262026
Ray MyersAI Engineer World's Fair 20252025
Marah Abdin, Robert McHardyAI Engineer World's Fair 20262026
Lou BichardAI Engineer Europe 20262026
The Prompt is the Platform

Metadata candidate

Dominik, Dominik TornowAI Engineer World's Fair 20262026
Beyang LiuAI Engineer World's Fair 20242024
Filip MakraduliAI Engineer World's Fair 20252025
Itamar FriedmanAI Engineer Code 20252025
Aparna DhinakaranAI Engineer Code 20252025
Stefania DrugaAI Engineer Summit 20252025
Geoffrey LittAI Engineer World's Fair 20252025
Michael ArnaldiAI Engineer Europe 20262026
Nicholas ArcolanoAI Engineer Code 20252025
Charles FryeAI Engineer World's Fair 20252025
Alex AlbertAI Engineer World's Fair 20242024
Dmitry PetrovAI Engineer World's Fair 20262026
Why Agent Engineering

Metadata candidate

swyx (Shawn Wang)AI Engineer Summit 20252025
Eugene CheahAI Engineer Summit 20252025
Your agent is blindfolded

Metadata candidate

Johan LajiliAI Engineer Europe 20262026
Zack ProserAI Engineer Europe 20262026
Olive SongEvent unknownUnknown
Tanmai GopalEvent unknownUnknown
Ford PriorEvent unknownUnknown
Dan Barrett, Raymond Weitekamp, Jose Montes de OcaEvent unknownUnknown
Anant SrivastavaAI Engineer Code 2025Unknown
Edo LibertyAI Engineer World's Fair 20262026
Max DrakeEvent unknownUnknown
Vlad LuzinAI Engineer World's Fair 20262026
Kay MalcolmEvent unknownUnknown
Dylan CouzonAI Engineer Code 2025Unknown
Matt GibiecAI Engineer World's Fair 20262026
Emile Baizel, Shruti AroraAI Engineer World's Fair 20262026
Fuad AliAI Engineer World's Fair 20262026
Jetashree RaviAI Engineer World's Fair 20262026
Javier GarzaAI Engineer World's Fair 20262026
Thomas Wolf, Olive SongEvent unknownUnknown
Jonathan KelleyEvent unknownUnknown
Yves Raimond, Jacqueline WoodAI Engineer World's Fair 20262026
Hossein NiazmandiAI Engineer World's Fair 20262026
Fuad AliEvent unknownUnknown
Matt PocockAI Engineer Code 2025Unknown
Zack Proser, Nick NisiEvent unknownUnknown
Charlie HoltzAI Engineer World's Fair 20262026
Ankur DuggalAI Engineer World's Fair 20262026
Aparna DhinakaranEvent unknownUnknown
Laurie VossAI Engineer World's Fair 2026Unknown
Mingsheng HongAI Engineer World's Fair 20262026
Paula DozsaEvent unknownUnknown
Gabriel Spencer-HarperAI Engineer Code 2025Unknown
Jack WotherspoonEvent unknownUnknown
Dylan Bristot, Sujee ManiyamAI Engineer Code 2025Unknown
Tejas KumarAI Engineer World's Fair 20262026
Brandon WaselnukEvent unknownUnknown
Elie BakouchAI Engineer World's Fair 20262026
Marc KlingenAI Engineer World's Fair 20262026
Morgan SuoEvent unknownUnknown
Eugene CheahEvent unknownUnknown
Ajay PrakashEvent unknownUnknown
Greg PstruchaAI Engineer World's Fair 20262026
Gabriel MartinezAI Engineer World's Fair 20262026
Laurie VossAI Engineer World's Fair 20262026
Eric SchwartzAI Engineer Code 2025Unknown
Dru KnoxAI Engineer World's Fair 20262026
Andrew OrobatorAI Engineer World's Fair 20262026
Laurie VossEvent unknownUnknown
Em ShreveAI Engineer World's Fair 20262026
Kevin OrellanaAI Engineer World's Fair 20262026
Jim ClarkAI Engineer World's Fair 20262026
The Highest Loop — swyx

Metadata candidate

Shawn "swyx" WangAI Engineer World's Fair 20262026
Peter WerryAI Engineer World's Fair 20262026
Zach LloydEvent unknownUnknown
Daniel KimAI Engineer World's Fair 20262026
Yunmo KooAI Engineer World's Fair 20262026
Tariq ShaukatEvent unknownUnknown
Nick Nisi, Zack ProserEvent unknownUnknown
Benjamin ClaviéEvent unknownUnknown
Andy Wong, Andrei Bocan, Shane WolfAI Engineer World's Fair 20262026
Jay Mok, Ben CoumesEvent unknownUnknown
Branislav BrzakAI Engineer NYC 2026 Online Talks2026
Charles FryeAI Engineer World's Fair 20262026
Jesse HanAI Engineer Summit 20232023
Archana Kamath, Tyler GillamAI Engineer World's Fair 20262026
Annabell Schäfer, Doneyli De JesusEvent unknownUnknown
Laurent Gil, Žilvinas UrbonasAI Engineer Code 2025Unknown
Zixuan LiEvent unknownUnknown
Gautam KorlamAI Engineer NYC 2026 Online Talks2026
Michael PattersonEvent unknownUnknown
Mike ChambersEvent unknownUnknown
Kent C. DoddsEvent unknownUnknown
Vlad LuzinEvent unknownUnknown
Jediah KatzEvent unknownUnknown
Lakshya AgrawalAI Engineer World's Fair 20262026
Lotte Verheyden, Annabell SchäferAI Engineer World's Fair 20262026
Paul BakausEvent unknownUnknown
Doug GuthrieAI Engineer World's Fair 20262026
Yoni Michael, Brandon CallenderAI Engineer World's Fair 20262026
Rowan ChristmasAI Engineer Code 2025Unknown
Manish Kapur, Addy OsmaniEvent unknownUnknown
Abhi AryaEvent unknownUnknown
Yuxuan ZhangAI Engineer Code 20252025
[REDACTED:username] NelsonAI Engineer Summit 20232023
Shawn "swyx" WangAI Engineer World's Fair 20262026
Nathan LambertAI Engineer World's Fair 20252025
Cedric VidalAI Engineer World's Fair 20252025
Uday Kiran Medisetty, Adam HudaAI Engineer World's Fair 20262026
Leonie MonigattiAI Engineer Europe 20262026
Mike SpitzAI Engineer Europe 20262026
Gabe De MesaAI Engineer World's Fair 20262026
Anita KirkovskaAI Engineer Summit 20252025
Zach Blumenfeld, Ben Squire, Ryan KnightAI Engineer World's Fair 20262026
Nick Nisi, Zack ProserAI Engineer World's Fair 20252025
Frank CoyleAI Engineer World's Fair 20262026
Henry MaoAI Engineer World's Fair 20252025
Abhishek BhardwajAI Engineer World's Fair 20252025
Kuba RogutAI Engineer Europe 20262026
Niklas NielsenAI Engineer Summit 20232023
Marlene MhangamiAI Engineer Europe 20262026
Filip KozeraAI Engineer World's Fair 20252025
Łukasz GandeckiAI Engineer World's Fair 20252025
Paul HenryAI Engineer World's Fair 20242024
Sunny MadraAI Engineer World's Fair 20242024
Greg BensonAI Engineer World's Fair 20252025
SallyAnn DeLucia, Fuad AliAI Engineer Code 20252025
Varun Badrinath Krishna, Petro Junior Milan, Rachelle MatternAI Engineer World's Fair 20242024
Build Systems, Not Code

Metadata candidate

Angie JonesAI Engineer World's Fair 20262026
Raj NavakotiAI Engineer Europe 20262026
Louis-François Bouchard, Paul Iusztin, Samridhi VaidAI Engineer Europe 20262026
Jeff NgAI Engineer World's Fair 20262026
John CrepezziAI Engineer Summit 20252025
Lou BichardAI Engineer World's Fair 20252025
Cedric VidalAI Engineer World's Fair 20252025
Harrison ChaseAI Engineer Summit 20232023
Kat Kampf, Ammaar ReshiAI Engineer Code 20252025
David KaramAI Engineer World's Fair 20252025
Apoorva JoshiAI Engineer World's Fair 20252025
Building Reactive AI Apps

Metadata candidate

Matt WelshAI Engineer Summit 20232023
Gergely OroszAI Engineer World's Fair 20262026
Michael FesterAI Engineer World's Fair 20252025
Rishabh GargAI Engineer World's Fair 20252025
Thariq ShihiparAI Engineer Code 20252025
Sunil PaiAI Engineer Europe 20262026
Jacob KahnAI Engineer Code 20252025
Codex and Subagents

Metadata candidate

Vaibhav Srivastav, Katia Gil GuzmanAI Engineer Europe 20262026
Codex, Behind the Harness

Metadata candidate

Dominik KundelAI Engineer World's Fair 20262026
Šimon PodhajskýAI Engineer Europe 20262026
Yusuf OlokobaAI Engineer Code 20252025
Priscila Andre de OliveiraAI Engineer Europe 20262026
Dhruv BatraAI Engineer World's Fair 20262026
Conquering Agent Chaos

Metadata candidate

Rick BlalockAI Engineer World's Fair 20252025
Content Is Code

Metadata candidate

Matt PalmerAI Engineer World's Fair 20262026
Santosh RadhaAI Engineer World's Fair 20242024
Hanchi WangAI Engineer World's Fair 20242024
Mahesh SathiamoorthyAI Engineer World's Fair 20262026
#define AI Engineer

Metadata candidate

Greg Brockman, swyx, Jensen HuangAI Engineer World's Fair 20252025
Zeke SikelianosAI Engineer World's Fair 20252025
Scott WuAI Engineer World's Fair 20252025
Dan ShipperAI Engineer Code 20252025
Ara KhanAI Engineer Europe 20262026
Kevin MaduraAI Engineer Code 20252025
Laurie VossAI Engineer World's Fair 20252025
Satya NittaAI Engineer World's Fair 20242024
Doug GuthrieAI Engineer World's Fair 20252025
Evals Are Not Unit Tests

Metadata candidate

Ido PesokAI Engineer World's Fair 20252025
Carlos Esteban, Doug GuthrieAI Engineer World's Fair 20252025
Sumaiya ShrabonyAI Engineer World's Fair 20262026
Maxime LabonneAI Engineer Europe 20262026
Alex Shaw, Ryan MartenAI Engineer World's Fair 20262026
Benjamin FletcherAI Engineer World's Fair 20242024
Field Guide to Fable

Metadata candidate

Thariq ShihiparAI Engineer World's Fair 20262026
fighting slop with slop

Metadata candidate

Vaibhav GuptaAI Engineer World's Fair 20262026
Nina Lopatina, Rajiv ShahAI Engineer World's Fair 20252025
Rafael LeviAI Engineer Europe 20262026
Brooke HopkinsAI Engineer World's Fair 20252025
Jason LopateckiAI Engineer World's Fair 20262026
Antje Barth, Mike ChambersAI Engineer World's Fair 20242024
Omri Bruchim, Tomer AstAI Engineer World's Fair 20262026
Jason LiuAI Engineer World's Fair 20262026
Gateways are All You Need

Metadata candidate

Karan SampathAI Engineer Europe 20262026
Omar SansevieroAI Engineer Europe 20262026
Git push, get an AI API.

Metadata candidate

Ryan Fox-TylerAI Engineer World's Fair 20242024
Dave Burnison, Alex Malebranche, Dimitrios Philliou, Christina Warren, Harald KirschnerAI Engineer World's Fair 20242024
GitHub Next Explorations

Metadata candidate

Rahul PanditaAI Engineer World's Fair 20242024
Sarah KhalifeAI Engineer World's Fair 20242024
Matija SosicAI Engineer Summit 20232023
Audry HsuAI Engineer Europe 20262026
Jonathan LarsonAI Engineer World's Fair 20252025
Brian JohnAI Engineer Code 20252025
Kyle KranenAI Engineer World's Fair 20252025
Mithun HunsurAI Engineer Summit 20232023
Tanmai GopalAI Engineer World's Fair 20242024
Phil HetzelAI Engineer Europe 20262026
Jia WuAI Engineer World's Fair 20262026
Eno ReyesAI Engineer World's Fair 20262026
Niels RoggeAI Engineer World's Fair 20262026
Ishan AnandAI Engineer World's Fair 20252025
Vasek MlejnskyAI Engineer World's Fair 20242024
Dan FengAI Engineer World's Fair 20262026
How to Build Trustworthy AI

Metadata candidate

Allie HoweAI Engineer World's Fair 20252025
Yegor Denisov-BlanchAI Engineer Code 20252025
Muktesh MishraAI Engineer World's Fair 20252025
Joe ReeveAI Engineer Europe 20262026
Ben KunkleAI Engineer Europe 20262026
Sally-Ann DeLuciaAI Engineer Europe 20262026
Ankur Goyal, Olmo MaldonadoAI Engineer World's Fair 20242024
Samuel ColvinAI Engineer World's Fair 20252025
Nicolas SchlaepferAI Engineer World's Fair 20242024
Intro to GraphRAG

Metadata candidate

Zach BlumenfeldAI Engineer World's Fair 20252025
AI Engineer Summit 20252025
Xiaofeng WangAI Engineer Summit 20252025
Ben HolmesAI Engineer World's Fair 20262026
Dat NgoAI Engineer Europe 20262026
Daniel WhitenackAI Engineer World's Fair 20242024
Joel BeckerAI Engineer Code 20252025
Daniel HanAI Engineer World's Fair 20242024
Joe FiotiAI Engineer World's Fair 20252025
Misha Kaletsky, Jonas TemplesteinAI Engineer Europe 20262026
Matt CareyAI Engineer Europe 20262026
Manuel OdendahlAI Engineer World's Fair 20252025
Ilan BigioAI Engineer World's Fair 20252025
Jess Grogan-Avignon, Jack WangAI Engineer Europe 20262026
Cedric VidalAI Engineer World's Fair 20242024
Mark HenningsAI Engineer Summit 20232023
Omar KhattabAI Engineer World's Fair 20252025
Garrett GalowAI Engineer Europe 20262026
Simon WillisonAI Engineer World's Fair 20242024
Simon WillisonAI Engineer Summit 20232023
Lech KalinowskiAI Engineer World's Fair 20262026
Jeronim MorinaAI Engineer World's Fair 20242024
DottaAI Engineer Europe 20262026
Antje BarthAI Engineer World's Fair 20262026
Pragmatic AI With TypeChat

Metadata candidate

Daniel RosenwasserAI Engineer Summit 20232023
Anish Agarwal, Matthew SchoenbauerAI Engineer World's Fair 20252025
Tengyu MaAI Engineer World's Fair 20252025
AI Engineer Europe 20262026
Raza HabibAI Engineer World's Fair 20242024
Will BrownAI Engineer World's Fair 20262026
Vaidas RazgaitisAI Engineer World's Fair 20262026
Boris StarkovAI Engineer Europe 20262026
Pamela Fox, Harald Kirschner, Gabriela de QueirozAI Engineer World's Fair 20242024
Gabriela de Queiroz, Aishwarya Srinivasan, Pamela FoxAI Engineer World's Fair 20242024
Scaffold Wisely

Metadata candidate

Rahul SengottuveluAI Engineer Summit 20252025
Preeti SomalAI Engineer World's Fair 20252025
Adrian BertagnoliAI Engineer Europe 20262026
Joshua SnyderAI Engineer Europe 20262026
Raahul Singh, Vanč LevstikAI Engineer World's Fair 20262026
Kyle Penfound, Jeremy Adams - CasañasAI Engineer World's Fair 20252025
Mike ChambersAI Engineer World's Fair 20252025
Laurie VossAI Engineer Europe 20262026
Pedro RodriguesAI Engineer Europe 20262026
Skills at Scale

Metadata candidate

Nick Nisi, Zack ProserAI Engineer Europe 20262026
State of Data

Metadata candidate

Sean CaiAI Engineer World's Fair 20262026
AI Engineer Summit 20252025
Josh PurtellAI Engineer World's Fair 20252025
Manish SanwalAI Engineer Summit 20252025
Stop Using RAG as Memory

Metadata candidate

Daniel ChalefAI Engineer World's Fair 20252025
Lars GrammelAI Engineer Summit 20232023
Tuomas Artman, Gergely OroszAI Engineer Europe 20262026
swyxAI Engineer Summit 20232023
Barr YaronAI Engineer World's Fair 20262026
The Agent-Native Company

Metadata candidate

Rick BlalockAI Engineer World's Fair 20252025
The AI Evolution

Metadata candidate

Mario RodriguezAI Engineer Summit 20232023
Chris White, Bryan Bischof, Brittany WalkerAI Engineer Summit 20232023
The Base Model is Dead

Metadata candidate

Varun SinghAI Engineer World's Fair 20262026
Travis FrisingerAI Engineer World's Fair 20252025
Brook RiggioAI Engineer World's Fair 20252025
Diamond BishopAI Engineer Summit 20252025
Addy OsmaniAI Engineer World's Fair 20262026
Junyang LinAI Engineer World's Fair 20252025
Alexander Embiricos, Romain Huet, Peter SteinbergerAI Engineer World's Fair 20262026
William LyonAI Engineer World's Fair 20252025
The Making of Devin

Metadata candidate

Scott WuAI Engineer World's Fair 20242024
The New Application Layer

Metadata candidate

Malte UblAI Engineer Europe 20262026
Kwindla Kramer, Kwindla Hultman KramerAI Engineer World's Fair 20262026
Arturo NunezAI Engineer World's Fair 20262026
The Pipeline Is Dead

Metadata candidate

Iris ten TeijeAI Engineer World's Fair 20262026
Jan CurnAI Engineer World's Fair 20252025
Jonathan MortensenAI Engineer World's Fair 20252025
Paul Klein IVAI Engineer World's Fair 20252025
Alex VolkovAI Engineer World's Fair 20262026
MuhtesemAI Engineer Summit 20252025
tldraw computer

Metadata candidate

Steve RuizAI Engineer World's Fair 20252025
Cormac BrickAI Engineer Europe 20262026
Forrest Brazeal, Matt BallAI Engineer World's Fair 20252025
Trust, but Verify

Metadata candidate

Shreya RajpalAI Engineer Summit 20232023
Paul Iusztin, Louis-François BouchardAI Engineer World's Fair 20262026
Audry HsuAI Engineer Europe 20262026
Jon PeckAI Engineer World's Fair 20252025
Chris LattnerAI Engineer World's Fair 20242024
João MouraAI Engineer World's Fair 20242024
Eugene YanAI Engineer World's Fair 20262026
Hassan El MghariAI Engineer World's Fair 20252025
Erik HanchettAI Engineer World's Fair 20262026
Matt DaileyAI Engineer World's Fair 20262026
Nico AlbaneseAI Engineer Summit 20252025
Vision: Zero Bugs

Metadata candidate

Johann Schleier-SmithAI Engineer Code 20252025
Lucas PalmaAI Engineer World's Fair 20262026
Remy GuercioAI Engineer Europe 20262026
Victoria MelnikovaAI Engineer World's Fair 20252025
Philipp SchmidAI Engineer Europe 20262026
Ahmad AwaisAI Engineer World's Fair 20252025
James LoweAI Engineer World's Fair 20252025
Kevin HouAI Engineer World's Fair 20252025
Balázs HorváthAI Engineer World's Fair 20262026
Tisha Chawla, Susheem KoulAI Engineer World's Fair 20262026
Mohak SharmaAI Engineer Summit 20252025
Lisa OrrAI Engineer Code 20252025
Garrett GalowAI Engineer World's Fair 20262026
Jemima AbuAI Engineer NYC 2026 Online Talks2026
Nicholas ArcolanoEvent unknownUnknown
Cole GannawayEvent unknownUnknown
Abdul DakkakAI Engineer World's Fair 20262026
Jose PalafoxAI Engineer World's Fair 20262026
Killian Carlsen-PhelanAI Engineer World's Fair 20262026
Ali-Reza Adl-TabatabaiAI Engineer World's Fair 20262026
Hariprasad SudharshanAI Engineer NYC 2026 Online Talks2026
Charlie Guo, Gabriel ChuaAI Engineer World's Fair 20262026
Wolfram RavenwolfAI Engineer World's Fair 20262026
Max Kanat-AlexanderEvent unknownUnknown
Jeff Koss, Leo PlatzerAI Engineer World's Fair 20262026
Ivan LeoAI Engineer World's Fair 20262026
Jan CurnAI Engineer Code 2025Unknown
Chris VelaEvent unknownUnknown
David AbitbolEvent unknownUnknown
Mike KriegerAI Engineer World's Fair 20262026
Mahmoud AbdelwahabAI Engineer NYC 2026 Online Talks2026
Philipp SchmidAI Engineer World's Fair 20262026
Kevin HouEvent unknownUnknown
Steve JamisonAI Engineer NYC 2026 Online Talks2026
Simran AroraAI Engineer World's Fair 20262026
Nilofer RajpurkarAI Engineer World's Fair 20262026
Ang LiAI Engineer World's Fair 20262026
Tariq Shaukat, Ali-Reza Adl-TabatabaiAI Engineer World's Fair 20262026
Sofia JakovcevicEvent unknownUnknown
Mark Lummus, Navinkumar PatilAI Engineer World's Fair 20262026
Michael GrinichAI Engineer World's Fair 20262026
Jiachen JiangAI Engineer NYC 2026 Online Talks2026
Dominik KundelAI Engineer World's Fair 20262026
Dean QuiñanolaAI Engineer World's Fair 20262026
Eli CohenAI Engineer World's Fair 20262026
Clare LiguoriAI Engineer World's Fair 20262026
Stephanie JarmakEvent unknownUnknown
Peter WerryAI Engineer World's Fair 20262026
Pedro LopezAI Engineer World's Fair 20262026

References

Coverage and source review
Processed transcripts
84 processed in full · 6 in the curated path
Automated source review
Passed
Metadata candidates
511 unreviewed; not verified topic membership
Corpus version
c76f4395d1050f81b2bfd27df900224591ef94b026792d1b65896097daa9f77c

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. Claude Code & the evolution of agentic coding

    https://www.youtube.com/watch?v=Lue8K2jqfKk&t=568s

  2. Claude Code in an IDE — Boris Cherny’s recorded edit

    https://ai.engineer/talks/Lue8K2jqfKk-claude-code-evolution-agentic-coding?view=slides#transcript-Lue8K2jqfKk-asset-20

  3. Future-Proof Coding Agents

    https://www.youtube.com/watch?v=wVl6ZjELpBk&t=176

  4. Function calling

    https://developers.openai.com/api/docs/guides/function-calling

  5. Building effective agents

    https://www.anthropic.com/engineering/building-effective-agents

  6. Language Models are Unsupervised Multitask Learners

    https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

  7. Agent mode — Cursor

    https://cursor.com/help/ai-features/agent

  8. Codex CLI — OpenAI

    https://developers.openai.com/codex/cli/

  9. Tools — OpenCode

    https://opencode.ai/docs/tools/

  10. Pi — official project

    https://pi.dev/

  11. Building pi in a World of Slop

    https://www.youtube.com/watch?v=RjfbvDXpFls&t=347s

  12. Introducing Devin — Cognition

    https://docs.devin.ai/get-started/devin-intro

  13. Introducing Codex — OpenAI

    https://openai.com/index/introducing-codex/

  14. GitHub Copilot on GitHub.com and GitHub Mobile

    https://docs.github.com/en/copilot/concepts/copilot-surfaces/copilot-on-github

  15. Risks and mitigations for GitHub Copilot cloud agent

    https://docs.github.com/en/copilot/concepts/security-governance-and-network-settings/risks-and-mitigations

  16. Feature Specs

    https://kiro.dev/docs/specs/feature-specs/

  17. Gherkin Reference

    https://cucumber.io/docs/gherkin/reference/

  18. Definition Of Refactoring

    https://martinfowler.com/bliki/DefinitionOfRefactoring.html

  19. How to Kill the Code Review

    https://www.youtube.com/watch?v=YgEv7IQzGdM&t=297

  20. Automating Large-Scale Refactors with Parallel Agents

    https://www.youtube.com/watch?v=rcsliSIy_YU&t=1674

  21. Configure permissions

    https://code.claude.com/docs/en/permissions

  22. IT Admin for the AI Workforce — Sarthak Aggarwal, Decawork

    https://www.youtube.com/watch?v=q-WOjZhOMCA&t=105

  23. Application of Theorem Proving to Problem Solving

    https://ijcai.org/proceedings/1969/22

  24. Combinatorial Sketching for Finite Programs

    https://people.csail.mit.edu/asolar/papers/asplos06-final.pdf

  25. Automating String Processing in Spreadsheets Using Input-Output Examples

    https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/popl11-synthesis.pdf

  26. Automatically Finding Patches Using Genetic Programming

    https://roars.dev/pubs/weimer2009automatically.pdf

  27. An Analysis of Patch Plausibility and Correctness for Generate-and-Validate Patch Generation Systems

    https://people.csail.mit.edu/rinard/paper/issta15.pdf

  28. On the Naturalness of Software

    https://web.cs.ucdavis.edu/~su/publications/natural.pdf

  29. Evaluating Large Language Models Trained on Code

    https://arxiv.org/html/2107.03374

  30. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    https://arxiv.org/abs/2310.06770

  31. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

    https://arxiv.org/abs/2405.15793

  32. OpenHands: An Open Platform for AI Software Developers as Generalist Agents

    https://arxiv.org/abs/2407.16741

  33. Software Development Agents: What Works and What Doesn't — Robert Brennan

    https://www.youtube.com/watch?v=o_hhkJtlbSs&t=147s

  34. LibCST documentation

    https://libcst.readthedocs.io/en/latest/

  35. Git Glossary

    https://git-scm.com/docs/gitglossary

  36. Git status documentation

    https://git-scm.com/docs/git-status

  37. AGENTS.md

    https://agents.md/

  38. pnpm install

    https://pnpm.io/cli/install

  39. unittest — Unit testing framework

    https://docs.python.org/3/library/unittest.html

  40. SWE-bench Evaluation Guide

    https://www.swebench.com/SWE-bench/guides/evaluation/

  41. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    https://arxiv.org/html/2310.06770v3

  42. Git Worktree Documentation

    https://git-scm.com/docs/git-worktree

  43. Cooking with Agents in VS Code

    https://www.youtube.com/watch?v=dyHpnnlkTc8&t=616

  44. Beyond permission prompts: making Claude Code more secure and autonomous

    https://www.anthropic.com/engineering/claude-code-sandboxing

  45. Repository map

    https://aider.chat/docs/repomap.html

  46. ripgrep user guide

    https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md

  47. LSP 3.17: Go to Definition

    https://raw.githubusercontent.com/microsoft/language-server-protocol/gh-pages/_specifications/lsp/3.17/language/definition.md

  48. LSP 3.17: Find References

    https://raw.githubusercontent.com/microsoft/language-server-protocol/gh-pages/_specifications/lsp/3.17/language/references.md

  49. LSP 3.17: Call Hierarchy

    https://raw.githubusercontent.com/microsoft/language-server-protocol/gh-pages/_specifications/lsp/3.17/language/callHierarchy.md

  50. Effective context engineering for AI agents

    https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

  51. The Oracle Problem in Software Testing: A Survey

    https://philmcminn.com/publications/barr2015.pdf

  52. How we built our multi-agent research system

    https://www.anthropic.com/engineering/multi-agent-research-system

  53. The Multi-Agent Architecture That Actually Ships — Luke Alvoeiro, Factory

    https://www.youtube.com/watch?v=ow1we5PzK-o&t=429

  54. Best practices for Claude Code

    https://code.claude.com/docs/en/best-practices

  55. Building Reliable Agentic Systems

    https://www.youtube.com/watch?v=bjNYEc908oQ&t=245

  56. Building Reliable Agentic Systems

    https://www.youtube.com/watch?v=bjNYEc908oQ&t=296

  57. Action Space: The Code Editor — Software Development Agents: What Works and What Doesn't

    https://ai.engineer/talks/o_hhkJtlbSs-software-development-agents-what-works-what-doesnt?view=slides#transcript-o_hhkJtlbSs-asset-9

  58. Git apply documentation

    https://git-scm.com/docs/git-apply

  59. Text editor tool

    https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool

  60. Edit formats

    https://aider.chat/docs/more/edit-formats.html

  61. What to look for in a code review

    https://google.github.io/eng-practices/review/reviewer/looking-for.html

  62. Node.js child process documentation

    https://nodejs.org/api/child_process.html

  63. Building Self-Coding Agents

    https://www.youtube.com/watch?v=Iw_3cRf3lnM&t=168

  64. Coding Evals: From Code Snippets to Codebases – Naman Jain, Cursor

    https://www.youtube.com/watch?v=tHN44yJoeS8&t=162

  65. QuickCheck: Automatic Testing of Haskell Programs

    https://hackage.haskell.org/package/QuickCheck

  66. Finding and Understanding Bugs in C Compilers

    https://users.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf

  67. Building AI Systems that Ship

    https://www.youtube.com/watch?v=vy7o1g2iHY8&t=306

  68. CI in the Era of AI: From Unit Tests to Stochastic Evals

    https://www.youtube.com/watch?v=WXy8Yy9xGss&t=191

  69. CI in the Era of AI: From Unit Tests to Stochastic Evals

    https://www.youtube.com/watch?v=WXy8Yy9xGss&t=271

  70. The Cure for the Vibe Coding Hangover

    https://www.youtube.com/watch?v=JsKTQbT58BY&t=2779

  71. Flaky tests

    https://docs.pytest.org/en/stable/explanation/flaky.html

  72. From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

    https://www.youtube.com/watch?v=qZj7sqmidqA&t=5878

  73. Git bisect documentation

    https://git-scm.com/docs/git-bisect

  74. A complex multi-file change, with debugging

    https://aider.chat/examples/complex-change.html

  75. From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization

    https://www.youtube.com/watch?v=JJGbw4ggaFs&t=456

  76. From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization

    https://www.youtube.com/watch?v=JJGbw4ggaFs&t=1258

  77. Effective harnesses for long-running agents

    https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

  78. Git diff documentation

    https://git-scm.com/docs/git-diff

  79. Software Development Agents: What Works and What Doesn't

    https://www.youtube.com/watch?v=o_hhkJtlbSs&t=702

  80. About protected branches

    https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches

  81. Git merge documentation

    https://git-scm.com/docs/git-merge

  82. Using Pre-trained Language Models to Resolve Textual and Semantic Merge Conflicts

    https://www.microsoft.com/en-us/research/uploads/prod/2022/07/issta22-merge-conflicts-llm.pdf

  83. Managing a merge queue

    https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue

  84. Temporal Activity Definition

    https://docs.temporal.io/activity-definition

  85. Demystifying evals for AI agents

    https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

  86. Agents reported thousands of bugs, how many were real? - Ian Butler and Nick Gregory

    https://www.youtube.com/watch?v=wAQK7O3WGEE&t=227

  87. SWE-bench Datasets

    https://www.swebench.com/SWE-bench/guides/datasets/

  88. SWE-rebench: Lessons from Evaluating Coding Agents on Real Software Engineering Tasks — Ibragim Badertdinov, Nebius

    https://www.youtube.com/watch?v=wcUJWP6WpGM&t=274

  89. Demystifying evals for AI agents

    https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

  90. Cross-validation: evaluating estimator performance

    https://scikit-learn.org/stable/modules/cross_validation.html

  91. Generalization in Adaptive Data Analysis and Holdout Reuse

    https://arxiv.org/html/1506.02629

  92. Quantifying infrastructure noise in agentic coding evals

    https://www.anthropic.com/engineering/infrastructure-noise

  93. SWE-bench evaluation grading implementation

    https://raw.githubusercontent.com/SWE-bench/SWE-bench/main/swebench/harness/grading.py

  94. RL for Autonomous Coding — Aakanksha Chowdhery, Reflection AI

    https://www.youtube.com/watch?v=QluDzKVfp6A&t=490

  95. Introducing SWE-bench Verified

    https://openai.com/index/introducing-swe-bench-verified/

  96. Why SWE-bench Verified no longer measures frontier coding capabilities

    https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/

  97. DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

    https://www.youtube.com/watch?v=Yk87oUPVaxU&t=67

  98. We are Changing our Developer Productivity Experiment Design

    https://metr.org/blog/2026-02-24-uplift-update/

  99. Amazon AGI

    https://www.youtube.com/watch?v=aeTb5BdmTTc&t=340

  100. ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, eBay

    https://www.youtube.com/watch?v=TJPInBjhE4Q&t=286

  101. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

    https://metr.org/Early_2025_AI_Experienced_OS_Devs_Study-paper.pdf

  102. How Coding Agents Change Software Development Forever - Hailong Zhang

    https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=238

  103. Agents need more than a chat

    https://www.youtube.com/watch?v=XNtkiQJ49Ps&t=589

  104. How to Test Your Agents on Real Tasks With Harbor - Xander Song, Arize AI

    https://www.youtube.com/watch?v=kRJjSOYqW0I&t=669

  105. SWE-rebench: Lessons from Evaluating Coding Agents on Real Software Engineering Tasks — Ibragim Badertdinov, Nebius

    https://www.youtube.com/watch?v=wcUJWP6WpGM&t=382

  106. Beyond the Prototype: Using AI to Write High-Quality Code

    https://www.youtube.com/watch?v=x_1EumTaXeE&t=670