AI Engineer World's Fair 2024

Copilots Everywhere

Thomas Dohmke· Chief Executive Officer, GitHubEugene Yan· Amazon18:22

Loading slides…

Read the talk

Copilots Everywhere: From Code Completion to Work Completed

Copilot’s evolution from ghost text to editable implementation plans shows how AI can reduce interruptions, expose hidden work, and leave developers more time to create.

From a talk by Thomas Dohmke and Eugene Yan

A model that could write JavaScript and Python

Could a language model write useful code simply from a spoken request? That question opens Thomas Dohmke’s account of Copilot’s origins in his conversation with Eugene Yan. Introduced as GitHub’s CEO at AI Engineer World’s Fair 2024, Dohmke brought a childhood fascination with programming in Germany to the job of overseeing Copilot’s launch.

Speaker card with portraits of Thomas Dohmke, CEO of GitHub, and Eugene Yan, Senior Applied Scientist at Amazon.Open full source frame
Thomas Dohmke and Eugene Yan, introduced at AI Engineer World’s Fair.

His first encounter happened on a Zoom call during the 2020 lockdown. GitHub engineers had access to an early Codex preview, which he recalls as a version of GPT-3. He tentatively remembers Oege, then leading GitHub Next, at the keyboard while colleagues dictated prompts. They asked for JavaScript, and the model put curly braces in the right places. They asked for Python, and it could write that too. The surprise was that this behavior came from a language model rather than a compiler applying a syntax tree.

Exploration became an autocomplete product, built while the team remained remote. For Dohmke, that history also answers the claim that innovation requires everyone to gather around the same whiteboard: Copilot emerged from engineers working in home offices. The next signal came from an internal preview, called a staff ship at GitHub. Dohmke recalls an internal engineer NPS of approximately 72–73, unusually high for an early product with hallucinations and an unfinished interface.

After the mid-2021 technical preview launched, the telemetry seemed surprising enough that he asked the team to check it again. Dohmke reports that Copilot initially wrote about 25% of the code in files where it was enabled, rising to about half by the interview. These are reported shares of code written, not measurements of task speed. He also recalls stronger acceptance and more lines written in languages such as Java. The team’s first two years brought successive discoveries about what models could do, before ChatGPT made those possibilities apparent to a much wider audience.

0:190:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:19 · section reference included

Why ghost text fit the developer’s workflow

As Copilot spread beyond the editor, Yan points to pull-request bots and asks how GitHub chose its experiments. Dohmke starts with the organization: daily changes in AI made a fixed feature backlog insufficient, even inside a company of roughly 3,000 people that was itself part of a much larger company. The product principle was more stable: build AI for engineers. GitHub’s own developers could use each improvement to make their work easier, including the work of building more Copilot features.

Putting AI into ghost text was a deliberate way to accommodate imperfect output. Autocomplete already had a familiar interaction contract in Google Docs, email, and IntelliSense: a suggestion might miss your intention; you keep typing or adjust it until something fits, then press Tab. The model did not need to be infallible for that interaction to be useful. Acceptance remained an explicit action by the developer.

For a small JavaScript illustration, imagine a developer has entered function isEven(n) { and receives a suggested body. Accepting the suggestion could produce:

javascript

function isEven(n) {
  return n % 2 === 0;
}

The suggested body is provisional until accepted, and the resulting code remains editable. The important interface decision is where the developer encounters the proposal: directly beside the code they are already writing.

Without that help, developers write until they get stuck, switch to a browser, search Google, Stack Overflow, or GitHub, then copy code back and modify it. Experienced developers may travel farther before that interruption; newcomers and occasional coders encounter it more often. Dohmke compares model errors with the problem of stale online answers: his Swift projects encounter new language, Xcode, and API versions after WWDC, so a once-correct answer may no longer apply. His comparison concerns changing context, not a measured equality of error rates.

Keeping the developer in flow is the product benefit. In another era, Dohmke suggests, the same feature might simply have been called smarter autocomplete. Its value comes from reducing repeated trips between editor and browser so the developer can finish the task.

3:493:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:49 · section reference included

From an issue to an editable implementation

Autocomplete was followed by conversation. After ChatGPT appeared in November 2022, GitHub introduced IDE chat and GPT-4 through its early-2023 preview work. A sidebar gave developers a place to ask questions, with retrieval-augmented generation, or RAG, drawing on available IDE context. That led to a larger workflow question: could assistance begin with the task itself, before the developer knew which code to edit?

Copilot Workspace approached that question by starting from a GitHub issue or an idea entered on GitHub, then connecting the task to its repository and an eventual pull request. The workflow described here is historical; its technical preview has since ended. Its central design was a sequence of editable representations of the change:

  1. Specification: examine current repository behavior and describe the desired behavior.
  2. Plan: identify how the relevant files need to change.
  3. Implementation: present the proposed changes as code in a diff.

At each stage, the human could change the bullet points or the code. A generated plan was something to inspect and revise, not an instruction that had to be accepted intact.

This addressed an earlier obstacle than typing speed: finding the implementation. Joining a project, contributing to an unfamiliar open-source repository, or returning from vacation can mean reconstructing where behavior lives among more than a thousand files. Workspace was intended to help developers move from current behavior to desired behavior while learning the codebase. Discovering that a feature also requires changing a test file is part of that assistance. The agent’s contribution includes helping the developer understand the work, with the developer still involved throughout.

7:267:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:26 · section reference included

A plan can reveal what a user story leaves out

The same process could help someone who did not write code. After the Workspace announcement at GitHub Universe the previous November, Dohmke recalls early feedback from a program or product manager: a user story could now be accompanied by a view of what implementing it would mean in the codebase. That makes the task description itself an object of investigation. Requirements often seem obvious until someone tries to turn them into specific changes.

Implementation scope also affects estimation. Dohmke invokes The Mythical Man-Month and a recent Pragmatic Engineer discussion, using a rule of thumb that actual work can take roughly twice the estimate. This is an estimation anecdote, not a measured result for Workspace. He places software alongside houses, roads, and other infrastructure as work people struggle to size. Workspace’s more immediate contribution is to expose whether a description is specific enough to yield a plan or code at all.

9:269:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:26 · section reference included

Learning in your own language, asking without embarrassment

Natural language also changes who can ask for help. Dohmke points to his T-shirt’s slogan, “Copilot speaks your language,” and describes models that can converse in almost any major human language. A beginner learning Boolean true and false need not first learn the concept in English: they could ask in German, Hindi, Brazilian Portuguese, Spanish, or Chinese.

His example begins with children growing up around phones, then encountering Super Mario or Minecraft and wanting to make a game or a web page themselves. Assistance in the language they already speak could shorten the path from that curiosity to programming. It could also make help available when parents lack a technical background or have run out of patience. A father of two, Dohmke jokes about eventually wanting to stop explaining the world and watch Netflix. The vision is access to patient help without requiring an English-speaking, technically trained household.

A similar barrier appears at work. New employees must learn institutional knowledge and local ways of doing things, but repeatedly asking colleagues basic questions can feel socially costly. Someone may worry that each question makes them look unqualified for the job. Dohmke sees copilots as a way to make company information easier to ask about, extending assistance beyond software developers to other workers learning an unfamiliar organization.

10:4110:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:41 · section reference included

An agent should wash the dishes

Asked what makes an agent, Dohmke reaches for a household appliance: an “AI dishwasher.” A human loads the inputs, the machine performs a job, and the human takes care of the output. In that broad sense, software already contains many agent-like systems.

SystemTrigger or inputWork performed
DishwasherLoaded dishesRuns a washing cycle
CI/CD, such as GitHub ActionsPull-request activityRuns configured jobs
Availability monitoringService signalsDetects trouble and pages someone

The monitoring example is concrete: GitHub should discover an outage and page staff before users have to report that they cannot reach their repositories. These systems are useful because they take responsibility for a bounded job.

Engineering still contains plenty of work that automation has not removed: boilerplate, security compliance, and dependency maintenance. Dohmke imagines a sunny Friday evening when a developer wants to join a barbecue but instead has to update Log4j dependencies. A security scanner that merely identifies the problem has added a task to the backlog. In the dishwasher analogy, it announces that the dishes are dirty and leaves the washing to someone else.

Developers often think of themselves as artists and creators, yet company and government requirements also demand necessary maintenance. Dohmke wants AI assistance that scans for security issues and helps fix them, returning time to feature development. Reviewable autofix suggestions were already an emerging capability around the event; his broader aspiration is for tools that take more of the unwanted work off the developer’s hands.

When Yan asks which agent he wants and how far away it is, Dohmke asks for one that burns down his security backlog. He gives no delivery timeline. The management problem is a real allocation constraint: security, compliance, and accessibility cannot consume all engineering capacity, but innovation cannot come at the expense of customer data and trust. There is no simple recipe for balancing those obligations; reducing the work itself would change the tradeoff.

He admits the familiar leader’s impulse to hear an estimate and think he could implement the feature faster himself—then explicitly rejects that thought. The estimate includes more than the visible coding task. Modern development carries many other process obligations, which is why he wants new abstraction layers across the development lifecycle. Faster code generation addresses only part of the work required to deliver a change.

13:1013:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:10 · section reference included

Make room for the enjoyment of building

For developers navigating these tools, Dohmke places AI alongside earlier changes he has lived through: a Commodore 64 and a PC, open-source software distributed on bookstore CDs and DVDs, then the internet, SourceForge, GitHub, and mobile development. Each transition opened new possibilities for building and collaboration. He sees AI as another such step, and encourages developers to “embrace our nerditude” and make new things.

That ambition has a personal scale as well as an organizational one. As CEO, he rarely gets to code; when a Sunday afternoon becomes available, he wants to spend it building. The host closes the session as he begins describing the maintenance that can consume that time, but Dohmke continues over the outro music: AI should bring enjoyment back to software development. His final encouragement is to build interesting things and have a good time doing it.

16:4016:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:40 · section reference included

Resources

Updates since the talk

Transcript download
Full transcript

Loading transcript…