AI Engineer World's Fair 2025

Building CISO-approved agent fleet architecture

Lou Bichard· Field CTO, Gitpod13:52

Loading slides…

Read the talk

Building CISO-approved agent fleet architecture

Gitpod’s architecture evolved from shared Kubernetes infrastructure to customer-hosted runners, balancing sensitive workloads, operational cost and the needs of autonomous agents.

From a talk by Lou Bichard

Before you start: Familiarity with cloud accounts, containers and virtual private networks will help; no prior knowledge of Gitpod is required.

Where should a secure development environment run?

How do you give developers a ready-to-use workspace while keeping sensitive work inside a regulated company’s infrastructure—and without making that company operate a complicated platform? That problem shaped Gitpod’s architecture before it became the foundation for an agent fleet. Lou Bichard, introducing himself as Gitpod’s Field CTO, traces roughly six years of architectural decisions, drawing on his background in software and platform engineering and nearly four years building automated development environments. The decisions matter to both vendors building secure tools and customers deciding which tools they can adopt.

A Gitpod development environment is the place where a developer works, not a staging or integration environment. Think of it as replacing the working environment on a laptop. Bichard reports that developers work inside Gitpod for around 36–37 hours per week. That makes availability consequential: an outage interrupts the customer’s ability to develop software, rather than merely taking an auxiliary tool offline.

The security requirements are equally central. Gitpod works with banks, pharmaceutical companies and healthcare organizations; restrictions on publishing customer names and logos limit the examples Bichard can identify. Serving those customers is a substantial undertaking for a small vendor with limited resources. By the time of the talk, that infrastructure also supported an agent offering that Bichard says had been announced two days earlier, after months of work with design customers.

“What is Gitpod?” slide with five points about standardized development environments, sensitive assets, regulated customers, weekly usage and downtime, beside an exploded stack illustration.Open full source frame
Gitpod’s development environments handle sensitive assets for regulated customers.

Each architectural iteration changed three things: customer ownership, operating cost and security. The journey began in 2019 with an internet-hosted, multi-tenant SaaS product. A customer could visit the website, enter a credit card and start using it. That simplicity did not meet the requirements of the secure enterprises Gitpod wanted to serve, leading first to self-hosting, then a managed customer-hosted model, and finally the architecture used for its agent fleet.

0:250:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:25 · section reference included

One click into a shared development environment

The original managed service had an immediately understandable attraction: click a link and arrive in a development environment. Bichard recalls a tweet from Shawn praising how quickly Gitpod delivered that first impressive experience. Underneath the interaction, Gitpod ran on GCP, chosen for its connection to Kubernetes, with development environments orchestrated as pods in shared, multi-tenant infrastructure.

Easy access also made free compute attractive to people mining cryptocurrency and otherwise abusing the service. Meanwhile, Bichard describes Kubernetes as a poor fit for Gitpod’s particular development workloads. The service was easy to try, but its shared infrastructure did not satisfy the enterprise requirements of professional developers working with real company systems and source code. Improving onboarding alone could not solve that deployment problem.

4:024:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:02 · section reference included

Self-hosting moves the operating burden

The obvious next step was to package the existing architecture as a self-hosted installation and give it to customers. Gitpod did exactly that. It opened support for Google infrastructure, with Bichard also recalling AWS support, without first tightly limiting the deployment configurations it would accept. Requests followed for different Kubernetes distributions and customer-specific configurations. Each variation widened the operating surface that customers and Gitpod had to understand.

Owning the infrastructure also meant owning its ongoing operation. Customers had to install the product and keep it running. Bichard illustrates the effect on return on investment with the possibility of assigning two or three people—or an entire team—to that work. The staffing burden consumes part of the value the product is supposed to create. This is the day-two problem: getting software installed is only the beginning of its cost of ownership.

Self-hosting also changed the relationship between vendor and customer. Gitpod had less direct contact with the people using and operating the product, even though adoption and rollout were critical to success. Bichard says some customers attempting adoption without Gitpod’s support were unsuccessful. The deployment model had moved the software into the customer’s boundary, but it had also made successful operation and adoption harder.

5:135:24
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:13 · section reference included

Manage the service, keep the workload in the customer account

Gitpod’s next model, called the substrate, separated hosting from management. Customers could keep the installation on their infrastructure while Gitpod managed the service. To make that supportable, Gitpod reduced deployment variance by committing to AWS as the infrastructure provider. The vendor could take back operational responsibility because it no longer had to accommodate every customer’s preferred platform configuration.

That arrangement established a deliberate information boundary:

Remained with the customerReturned to Gitpod
Development workloadsOperational telemetry
Source code and dataInformation needed to manage the service
Integrations

The workload and its sensitive assets stayed on customer infrastructure, while a small amount of operational information flowed to the vendor. That division met the requirements of the regulated customers Bichard describes.

Operationally, the model was a success, but it preserved the complexity of the underlying Kubernetes architecture. Installation remained difficult, and multiple clusters imposed a high fixed cost. Reducing the number of supported configurations had made the system easier to manage; it had not removed the infrastructure that made it expensive and complicated in the first place.

Slide lists a BYOC model, AWS-only infrastructure and managed operations, followed by installation difficulties and two-cluster expense, beside a New Stack article screenshot.Open full source frame
The substrate model reduces variance but retains installation overhead and the expense of two clusters.
6:487:05
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:48 · section reference included

Replace the clusters with a small runner

Gitpod ultimately moved away from Kubernetes for this use case. Its engineering account, We’re leaving Kubernetes, develops the workload-specific reasoning behind that decision. The design question became more direct: what is the smallest architecture that can run securely inside regulated companies without imposing a large operational burden on them?

The replacement uses a runner, analogous to the runner an engineer might know from GitHub CI. It keeps source code and access to customer data on the customer’s infrastructure. In the AWS architecture described in the recording, the runner is a single ECS task running one container. Bichard puts the runner’s cost at single-digit dollars per month, separate from the development workloads it launches. This is the historical design described in the talk, not a complete specification for a current deployment.

The runner launches development environments using the capabilities of its host platform. On AWS, those workloads run on EC2 and are backed up to EBS. The same principle applies to the Linux runner: inherit the qualities of the platform on which it runs. Gitpod chose provider-native functionality over Kubernetes’ theoretical portability, avoiding the additional abstraction and operational complexity that portability had brought to its workload.

The remaining question was what Gitpod could operate centrally so customers would not have to. The vendor retained coordination metadata, including IDs and user-related identifiers, while the source code and valuable intellectual property stayed inside customer infrastructure. Bichard characterizes that metadata as non-personally-identifying and less sensitive to these customers than their source code; that is a description of the intended data boundary, not a general guarantee that any user identifier is harmless.

Component or dataLocation in the described architecture
Management metadata and user IDsGitpod’s side
RunnerCustomer infrastructure; ECS on AWS
Development environmentsCustomer infrastructure; EC2 on AWS
Workload backupsEBS on AWS
Source code and data accessCustomer infrastructure

The customer keeps the sensitive workload; the vendor keeps much of the coordination burden. This is also the foundation on which Gitpod runs its agent fleet.

“Thinking from first principles” architecture diagram showing a Gitpod management plane, customer-account runner and environments, a separate customer-resources account, and customer roles linked by arrows.Open full source frame
Gitpod’s management plane connects to a runner and environments in the customer account.
8:018:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:01 · section reference included

Installation is simple once the network is ready

The deployment video makes the boundary visible in the user experience. A user selects a workload, but execution happens inside the customer’s network and cloud account. The runner setup follows a short sequence:

  1. Use the runner interface to create a CloudFormation configuration.
  2. Supply the customer’s network details on the customer’s side.
  3. Create the installation in that account.

The interface makes provisioning approachable without moving the workload out of the customer’s infrastructure.

Bichard says runner installation can take as little as three minutes once the required network details are ready. The usual bottleneck is obtaining the VPC and subnet information, identifying who owns those resources, and finding someone authorized to allocate them. The claimed installation time therefore describes provisioning, not the entire enterprise approval and onboarding process. A simpler deployment reduces technical overhead, but it does not eliminate the organization’s responsibility for its network.

10:1710:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:17 · section reference included

Give agents the environment developers already use

The agent offering grew out of work with customers and design partners dating back to the start of the year. Because Gitpod already had infrastructure inside those customers’ accounts, it could reuse the same development workloads. An autonomous coding agent needs much of what makes a human developer productive: source code, a place to execute and iterate, and access to the internal systems involved in the task.

Those internal systems might include a database or a cluster. Within the existing development environment, an agent can work on source code with the access and privileges available to an individual developer. The architecture provides an execution location and an access boundary for the agent rather than requiring a separate external workspace with a new path into customer systems.

That reuse underpins the offering’s privacy-first positioning. The agent runs inside customer infrastructure, building on the same deployment model developed for regulated organizations. The product panel presents Ona as the software engineering agent hosted in the customer’s VPC.

Gradient Ona panel describing enterprise hosting, auditability, LLM support, autonomous parallel tasks and IDE takeover, with ona.com credited below.Open full source frame
Ona is presented as a privacy-first software engineering agent hosted in your VPC.
11:2111:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:21 · section reference included

Make platform activity auditable

The redesign also gave Gitpod an opportunity to build the platform around an API-first interface. Bichard says platform actions and interactions—including creating a development environment—are audit logged, giving customers a history of activity on the platform. That audit history extends to agent tasks when they use the same infrastructure. The scope here is platform operations and agent-task activity, rather than a claim that every shell command or file access inside an environment is recorded.

For buyers of AI tools, the underlying architecture therefore belongs in the purchasing decision: where work executes, which infrastructure the customer owns, and what operational qualities come with that arrangement. For vendors, Gitpod’s experience ties technical simplification directly to the customer’s ability to adopt and operate the product. In this case, moving away from a complicated Kubernetes deployment made it possible to preserve the customer’s security boundary while reducing the burden of running the service.

12:2512:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:25 · section reference included

Resources

From the talk

  • Gitpod’s engineering account of the security, storage and resource-management difficulties behind its departure from Kubernetes.

Updates since the talk

Transcript download
Full transcript

Loading transcript…