AI Engineer World's Fair 2024
GitHub's AI-Powered Security Platform
Loading slides…
Read the talk
GitHub’s AI-Powered Security Platform
CodeQL analysis, secret scanning, and AI-generated fixes bring application security into the pull request, where developers can investigate, review, and verify changes before merging.
From a talk by Sarah Khalife
Before you start: Familiarity with Git commits, pull requests, dependencies, and basic JavaScript will help with the examples.
How can AI improve application security?
How can AI help developers secure the software they build? Questions about AI security often center on privacy, legal exposure, and protecting AI systems themselves. Here, the problem is application security: using AI to improve the tools developers already depend on to find and fix vulnerabilities. Copilot Workspaces, Enterprise, and Futures provide the opening context, but the ambition extends across GitHub’s collaboration, productivity, and security workflows, including pull requests and code review.
That platform approach also needs integration points. Khalife describes the capabilities as generally API-backed or supported by third-party integrators, allowing organizations to assemble a solution around their existing tools rather than depend on one prescribed path.
Khalife’s potentially outdated slide reports over 100 million registered developers, four million organizations, and 90% of Fortune 100 companies using GitHub. The practical significance of that reach is the feedback loop: customers and community members influence what GitHub builds next.
Sarah Khalife, a principal solutions engineer at GitHub, describes about four and a half years at the company, spanning the period before Copilot and nearly before Actions. Her customer work at the time focuses on financial services, following work across other enterprise industries. That customer-facing perspective makes the recurring question concrete: how do developers, security teams, vendors, and partners improve the same daily workflow?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give security analysis a place in development
GitHub Advanced Security embeds AppSec in the developer platform, using the familiar GitHub experience. The progression starts with the existing security capabilities, adds AI assistance, and then follows those capabilities into a live workflow. The organizing principle is to make security part of everyday development.
Code scanning detects vulnerable patterns through static application security testing, or SAST. CodeQL supplies analysis that follows data through code, identifying sources and the paths that make a vulnerability possible. That context matters twice: first when a developer investigates a finding, and later when AI needs enough information to propose a repair.
Teams can write custom queries and draw on community query packs; Khalife names Microsoft, Google, and Uber alongside GitHub’s own contributions. The component distinction is useful here: the public CodeQL libraries and queries are separate from the CLI, which has its own license terms. Describing the whole toolchain as open source would blur that distinction.
CodeQL is presented as the SAST component, with native container scanning outside the near-term plan at the time of the talk. Other scanners can still contribute to the same developer workflow through SARIF, a format for exchanging analysis results.
| Analysis | Route into GitHub code scanning |
|---|---|
| Source-code vulnerabilities | CodeQL analysis |
| Container vulnerabilities | Third-party results through SARIF |
| Infrastructure-as-code findings | Third-party results through SARIF |
The integration makes those results available in pull-request feedback, so developers can encounter findings while they are already reviewing a change.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Prevent exposure, then respond to what escaped
A developer leaves an AWS or Azure key in a log or test file, forgets to exclude the file with .gitignore, and pushes it. The consequence can be an attacker using the credential to spin up cryptocurrency miners. Secret scanning looks for API keys, custom secrets, and plaintext credentials across repositories; the harder plaintext cases motivate the later AI features.
Push protection moves detection to the boundary where code is sent to GitHub. Blocking a push can prevent the credential from reaching the remote repository. It does not prevent local commits from existing: the command-line workflow requires removing the secret from affected local commits before retrying. Khalife’s point about avoiding revocation applies when the block has prevented exposure and the credential has not otherwise leaked.
Once a credential has been exposed, deleting a line is not enough. Scanning can find secrets in Git history and in supported issue, pull-request, and comment content. Khalife recommends revoking exposed credentials, and cautions against deleting historical records in regulated environments where audit requirements matter. That is a response to already exposed secrets, distinct from cleaning up unpublished local commits after a blocked push.
Supply chain security adds another category: vulnerable dependencies. Dependabot identifies affected dependencies so teams can consider upgrading to a version that resolves the vulnerability. Additional AI capabilities for this area are described as early-stage work, without a detailed demonstration.
For recognizable vendor credentials, detection quality also depends on how the credential is designed. GitHub incorporates partner patterns and encourages vendors to add hashes and more identifying structure, making secrets easier to distinguish from unrelated strings. Khalife loosely estimates detection near 99% or “ninety-nine point nine something percent” for high-fidelity partner patterns, while explicitly saying she does not know the average. No metric definition accompanies that estimate; the mechanism is the additional structure that reduces false positives.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn a finding into a repair proposal
Security becomes a coordination problem when AppSec asks for ten vulnerabilities to be fixed by tomorrow and a developer does not yet understand the vulnerabilities. Shared responsibility requires more than handing over an alert list: the platform needs to help both sides understand the problem and move toward a repair.
The proposed AI benefits follow that sequence: easier identification, faster remediation, and improved productivity. Finding vulnerabilities without fixing them leaves the central problem unresolved. Earlier remediation also reduces the chance that a problem is discovered only after deployment or customer exposure.
Code scanning autofix puts a suggested repair directly in the pull request. The developer can inspect it, edit it, or commit it and rerun tests. CodeQL supplies the vulnerability’s data flow and information about common fixes; the backend uses that context to prompt the AI system. The suggestion therefore starts from an analyzed finding rather than an isolated request to make code secure.
Khalife informally estimates that autofix was successful in perhaps 70% of use cases among customers who tested it. The customer sample and success criteria are unspecified. Autofix is in public beta in this recording; it subsequently became generally available as Copilot Autofix in August 2024.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use examples and context to find secrets
Writing a regex is one task; trusting it across an organization’s repositories is another. AI-assisted custom pattern generation starts with samples of the secrets a team wants to detect. It produces a candidate pattern that can serve as a starting point, and additional examples give it more context for refinement.
The benefit is partly integration. Developers already seek regex help through ChatGPT, Copilot in an IDE, or search-engine cheat sheets. Bringing that assistance into secret scanning reduces the work needed to translate examples into a usable detection pattern. It generates the pattern for finding a secret, not a replacement credential.
Unstructured passwords pose a different problem. An assignment such as password = ... does not establish that the value is a real credential, much less that someone is exploiting it. AI analyzes repository context to identify likely true positives. In the initial version described here, these findings appear under other, acknowledging that false positives remain.
For confirmed exposed passwords, the next steps are revocation or rotation and storage in a secret manager such as HashiCorp Vault or Azure Key Vault. Internal company passwords are a particularly useful target because they need not follow a vendor’s recognizable token format. This AI password feature should not be assumed to inherit every secret-scanning surface: its subsequent July 2024 public beta covered Git content, excluding issue and pull-request content and push protection. The intended workflow still connects identification to a concrete response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the organization’s security workload
The live demonstration starts in an organization’s Security tab, available to administrators and the security manager role. Advanced Security provides richer organization-wide information; without it, some Dependabot information remains available.
The demo environment contains approximately 73,000 alerts; Khalife explicitly identifies it as nonproduction code that will not be deployed to Azure. The overview brings together secrets, SAST findings, third-party scanner results, and dependency vulnerabilities. Those categories help turn a large total into identifiable kinds of work.
The dashboard then adds time and repository context: how old alerts are, how long remediation takes, how often findings are resolved, and which repositories are affected. The selected view shows remediation cards, alert activity, and repository impact analysis together. This lets a security team examine the behavior of its backlog, not just its size.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Find the repositories your pipeline misses
An alert dashboard cannot reveal vulnerabilities in repositories that were never scanned. A team may believe its CI pipeline provides full coverage, while small automation repositories sit outside that pipeline entirely. If scanning only happens during CI, those repositories can remain invisible to the security process.
The coverage view separates secret scanning, code scanning, and Dependabot. In the demo organization, secret-scanning coverage is 99%, while code-scanning coverage is 57%. Khalife associates the broader secret-scanning coverage with easy, often one-click enablement. The lower code-scanning figure prompts investigation: what is missing, and which additional repositories should be scanned?
This organization-level view serves administrators and security managers. The demonstration then switches to the developer’s view: a repository’s own Security tab, where findings become specific pieces of work against that codebase.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Trace multiple paths to a common repair point
The repository’s Code Scanning alerts represent reactive work: vulnerabilities have already been found, and the team needs to reduce a backlog. Opening a finding reveals the vulnerability; selecting Show Paths traces the flow from its source to its sink, the point where the unsafe operation occurs.
A single sink can be reachable through more than one exploit path. In the demonstration, examining step eight reveals a different source that reaches the same sink. Khalife then proposes fixing the vulnerability at step seven, a common point across those paths. The useful question is not merely which line was flagged, but where a change could address the shared cause of multiple routes to exposure. This remains an investigation and proposed repair location; the next example moves prevention into the pull request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Review the XSS fix before merging
The pull request introduces a cross-site scripting vulnerability. Advanced Security explains the vulnerability, while autofix proposes a change specific to the affected code. The feedback arrives before the pull request is merged into a main or development branch, keeping diagnosis and repair inside the same review workflow.
The suggestion calls for installing and importing escape-html. Its relevant operation is escaping untrusted text before inserting it into HTML. For example, this JavaScript renders a supplied name as text inside a paragraph:
javascript
const escapeHtml = require('escape-html');
function renderGreeting(name) {
return `<p>Hello, ${escapeHtml(name)}</p>`;
}
const name = '<img src=x onerror=alert(1)>';
console.log(renderGreeting(name));
// <p>Hello, <img src=x onerror=alert(1)></p>
The library is context-specific: it supports ordinary HTML text and quoted attribute values, not JavaScript or CSS bodies. The proposed repair therefore still requires checking where the value is used.
Khalife describes the potential effort saved as ranging from minutes to hours or days, depending on the developer’s familiarity with the pull request, code, and vulnerability; no elapsed-time measurement is supplied. She still reads and analyzes the suggestion to decide whether it is appropriate.
The review sequence is concrete:
- Inspect the proposal. Read the explanation and the code and dependency changes.
- Decide whether to accept or edit it. The developer remains responsible for the repair.
- Commit the fix. The commit triggers scans again so the team can check whether the vulnerability has been remediated.
The selected frame shows proposed changes to index.js and package.json, a dependency summary, and Dismiss, Edit, and Commit fix controls. The recording describes the verification step without establishing a passing result.
That is the demonstrated endpoint: an analyzed vulnerability becomes a reviewable proposal, with a path back through scanning before merge. AI-assisted secret-pattern generation and additional secret detection remain separate demonstrations offered afterward at the Microsoft booth.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Explore the libraries and queries used for CodeQL security analysis.
A small JavaScript library for escaping text inserted into HTML, with installation instructions and context-specific usage limits.
Updates since the talk
Current instructions for removing detected secrets from local commits before pushing again.
Full transcript
Loading transcript…