AI Engineer World's Fair 2024

GitHub Next Explorations

Rahul Pandita· Automated Software Engineering Researcher, GitHub18:18

Loading slides…

Read the talk

GitHub Next Explorations

Factory electrification offers a useful lens for two GitHub Next explorations: predicting related edits and helping developers turn an issue into verified code.

From a talk by Rahul Pandita

Before you start: Basic familiarity with functions, GitHub issues, pull requests, and terminal commands will help you follow the demonstrations.

What changes when the machinery changes?

How much of a workflow exists to accommodate the technology underneath it? That question frames GitHub Next’s search for new ways to develop software. Rahul Pandita describes a team of about 20 researchers, senior developers, and tool builders working outside regular product development and reporting directly to GitHub’s CEO. Its purpose is to explore possibilities, then pass the learning to product teams building tools such as Copilot.

Pandita introduces the problem through Andrew Ng’s analogy of AI as the new electricity. Before factory electrification, a central steam engine drove large shafts, which drove auxiliary shafts, which workers connected to through belts and pulleys. The power system dictated the factory’s layout. Work had to happen where mechanical power could reach it; workers organized themselves around the machinery.

Slide with Andrew Ng’s quotation about AI transforming industries beside an electrical panel illustration.Open full source frame
Andrew Ng’s “AI is the new electricity” analogy.

Small electric motors changed that constraint because they retained their efficiency at smaller sizes. Power could be distributed around the factory, making a different floor plan possible. Yet the new layout did not appear automatically with the new motor. In Pandita’s historical framing, roughly forty years separated early electric motors in the 1880s from mainstream adoption in the 1920s. He uses that interval to emphasize experimentation: finding useful applications, improving them, and reducing risk until using the technology becomes normal practice. A new capability creates room for a new workflow; discovering that workflow is separate work.

0:280:50
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:28 · section reference included

Testing an idea before making it a product

If the right AI development workflow were already known, the efficient move would be to build it. Uncertainty makes rapid prototypes useful instead. GitHub Next puts an idea through progressively broader feedback:

  1. Build a functional prototype and use it heavily within Next.
  2. Expand internal use to the rest of GitHub if it survives that first test.
  3. Release a technical preview to early adopters and learn from their use.
  4. Consider productization if the evidence continues to support it.

These are gates, not a guaranteed release pipeline. An exploration can be stopped or shelved at any stage when the signals do not justify continuing.

Copilot began as one of these experiments. Other explorations included Copilot for CLI, Copilot Voice, GitHub Blocks, and SpecLang. Some became products, some contributed capabilities to existing products, and others were shelved because the timing or the idea was wrong. Shelving an implementation does not discard what the team learned from it.

The two explorations examined here are Next Edit Suggestions and Copilot Workspace. The recording captures them as active explorations in 2024; the current GitHub Next catalog labels Next Edit Suggestions as Product and Workspace as Completed. The demonstrations below follow the workflow shown in the recording.

3:594:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:59 · section reference included

An edit prediction also needs a navigation interface

Predicting a useful change does not settle how to present it. At the time of the talk, the team was still investigating several connected problems:

  • Presentation: Is ghost text the right interface for replacing or deleting existing code?
  • Navigation: How should a suggestion appear when its location is outside the viewport, or in a file that is not open?
  • Accuracy: How should models be fine-tuned for this particular task so that suggestions are relevant enough to help?

The location problem expands the interaction beyond autocomplete: the developer must understand both the proposed change and where it belongs.

Internal dogfooding remained the release gate. Pandita anticipated an IDE appearance in the following months, either through a standalone Next technical preview or within an existing Next product, conditional on those investigations producing a satisfactory experience. That was a forecast, not an announcement that the demonstrated behavior was already available.

7:398:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:39 · section reference included

Expanding assistance to the development inner loop

Related edits still operate at the level of code. Moving to task completion means helping with the reasoning that precedes implementation. A developer starts with an assignment, works out what it requires, examines the repository’s languages and frameworks, and determines how new behavior can fit the existing code. The desired result then has to become a concrete set of file changes.

StageQuestion to resolveArtifact
TaskWhat am I trying to accomplish?Defined problem
SpecificationWhat exists, and what should exist?Current and desired states
PlanWhich files need which changes?Implementation plan
ImplementationHow do those changes become code?Modified files

Code is one artifact in this loop, not the entire loop. Copilot Workspace aims to make AI useful across these stages rather than limiting assistance to the final act of writing code.

Workspace itself emerged from smaller explorations. SpecLang investigated natural language as a functional specification for programs. A separate effort investigated improving completions by re-prompting models with runtime information. Those experiments, combined with user feedback, became a broader environment for completing tasks.

Developer interviews exposed several requirements. Getting started on an assigned issue was difficult. Trusting AI output was another obstacle. Developers also wanted to retain the problem-solving part of their work: AI should act as a thought partner they could collaborate with, while they remained in control.

Workspace maps those needs to concrete interactions. A one-click proposal provides a starting point. A built-in runtime lets the developer check generated code. Editable intermediate artifacts allow course corrections, and collaboration can happen through a GitHub pull request or by sharing the workspace itself with colleagues.

8:458:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:45 · section reference included

From a missing Rust example to a file-level plan

The Workspace demonstration uses Monaspace, another GitHub exploration: a family of coding fonts with a website demonstrating their features. Its playground shows syntax highlighting across languages, but Rust is missing. The task is specific and observable: add a Rust example to that playground.

Pandita starts from a GitHub issue requesting that addition and opens it in Workspace. The interface presents a specification divided into Current and Proposed states. The specification, plan, and implementation shown during this walkthrough are cached to keep the demonstration within its allotted time. Pandita describes uncached generation as taking minutes rather than hours; the fast transitions on screen are not a latency measurement.

The current specification identifies missing Rust support. The proposed specification describes the target: add the syntax-highlighting package, include it in package.json, and update the related files. This gives the developer something to assess before asking for code. Does the system understand the gap, and does its proposed destination match the issue?

Copilot Workspace specification panel for the Monaspace Rust example issue, with Current and Proposed lists and a Generate plan button.Open full source frame
Copilot Workspace shows current and proposed specifications for adding a Rust example.

The generated plan identifies three files to update. Pandita reviews that plan, judges it to be at a useful level of detail, and requests implementation. Workspace then displays the resulting file changes. The progression matters: an issue becomes a description of the desired behavior, that description becomes a file-level plan, and the reviewed plan becomes an implementation.

12:0012:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:00 · section reference included

Correcting the proposal and checking the result

The generated artifacts remain editable. Pandita points to instructions for adding Rust to the language mappings and adding code documentation, then shows that chat can also target a particular location for revision. He does not run that regeneration path during the presentation because it would leave the cached sequence and require another iteration. The interaction is available for steering the work; the recording does not demonstrate its regenerated result.

Verification moves the task from plausible file changes to observable behavior. In the integrated terminal, Pandita asks to install and run the repository. The first synthesized command is unsuitable. He edits it to the command he wants before executing it in an actual terminal. This correction is part of the workflow: natural language proposes a command, but the developer still inspects what will run.

The terminal runs the corrected command, and Pandita reports that the code compiles. He then opens a live preview, which takes longer to load than its displayed estimate suggests. Compilation provides one check; seeing the requested change in the application provides another.

Why generate a simple npm command instead of typing it? On a desktop, direct entry may be easier. On a phone, command syntax and symbols are more tedious to enter through a mobile keyboard. Natural-language command synthesis addresses that input constraint, allowing a developer to work from a mobile device without manually composing every shell command. The preview eventually loads with a Rust example, completing the demonstrated path from issue to a running preview—not to a production deployment.

14:2114:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:21 · section reference included

Learning and review become the next workflow questions

Workspace’s continuing improvements include runtime support for synthesizing terminal commands and faster file completions. Beyond Workspace, Next is investigating how developers learn with AI and how code review should change if AI writes most of the code. The closing research slide also places editors, terminals, planning, and tracking within that broader agenda. Some investigations may reach technical preview; others may be stopped when they fail to find a useful direction.

Other Next Explorations slide listing developer learning with AI, the editor and terminal, planning and tracking, and code review when most software is written by AI.Open full source frame
GitHub Next’s research questions span learning, editors, planning, and code review.

That returns the discussion to the factory floor. The opportunity is larger than inserting a new source of power into an unchanged arrangement of work. It includes discovering where developers should inspect a proposal, revise a plan, navigate to a related edit, or run a check. Pandita closes by inviting collective experimentation to shorten the distance between AI’s capabilities and everyday workflows that make those capabilities useful.

16:4216:53
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:42 · section reference included

Resources

From the talk

  • GitHub Next's exploration of maintaining program behavior in natural-language specifications while a toolchain manages implementation.

  • Five compatible coding typefaces, with font downloads, installation instructions and typography features.

  • Paul A. David's May 1990 study of factory electrification, organizational change and delayed productivity gains.

Transcript download
Full transcript

Loading transcript…