AI Engineer World's Fair 2025
Building CISO-approved agent fleet architecture
Loading slides…
Read the talk
Building CISO-approved agent fleet architecture
Gitpod’s architecture evolved from shared Kubernetes infrastructure to customer-hosted runners, balancing sensitive workloads, operational cost and the needs of autonomous agents.
From a talk by Lou Bichard
Before you start: Familiarity with cloud accounts, containers and virtual private networks will help; no prior knowledge of Gitpod is required.
Where should a secure development environment run?
How do you give developers a ready-to-use workspace while keeping sensitive work inside a regulated company’s infrastructure—and without making that company operate a complicated platform? That problem shaped Gitpod’s architecture before it became the foundation for an agent fleet. Lou Bichard, introducing himself as Gitpod’s Field CTO, traces roughly six years of architectural decisions, drawing on his background in software and platform engineering and nearly four years building automated development environments. The decisions matter to both vendors building secure tools and customers deciding which tools they can adopt.
A Gitpod development environment is the place where a developer works, not a staging or integration environment. Think of it as replacing the working environment on a laptop. Bichard reports that developers work inside Gitpod for around 36–37 hours per week. That makes availability consequential: an outage interrupts the customer’s ability to develop software, rather than merely taking an auxiliary tool offline.
The security requirements are equally central. Gitpod works with banks, pharmaceutical companies and healthcare organizations; restrictions on publishing customer names and logos limit the examples Bichard can identify. Serving those customers is a substantial undertaking for a small vendor with limited resources. By the time of the talk, that infrastructure also supported an agent offering that Bichard says had been announced two days earlier, after months of work with design customers.
Each architectural iteration changed three things: customer ownership, operating cost and security. The journey began in 2019 with an internet-hosted, multi-tenant SaaS product. A customer could visit the website, enter a credit card and start using it. That simplicity did not meet the requirements of the secure enterprises Gitpod wanted to serve, leading first to self-hosting, then a managed customer-hosted model, and finally the architecture used for its agent fleet.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Self-hosting moves the operating burden
The obvious next step was to package the existing architecture as a self-hosted installation and give it to customers. Gitpod did exactly that. It opened support for Google infrastructure, with Bichard also recalling AWS support, without first tightly limiting the deployment configurations it would accept. Requests followed for different Kubernetes distributions and customer-specific configurations. Each variation widened the operating surface that customers and Gitpod had to understand.
Owning the infrastructure also meant owning its ongoing operation. Customers had to install the product and keep it running. Bichard illustrates the effect on return on investment with the possibility of assigning two or three people—or an entire team—to that work. The staffing burden consumes part of the value the product is supposed to create. This is the day-two problem: getting software installed is only the beginning of its cost of ownership.
Self-hosting also changed the relationship between vendor and customer. Gitpod had less direct contact with the people using and operating the product, even though adoption and rollout were critical to success. Bichard says some customers attempting adoption without Gitpod’s support were unsuccessful. The deployment model had moved the software into the customer’s boundary, but it had also made successful operation and adoption harder.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Manage the service, keep the workload in the customer account
Gitpod’s next model, called the substrate, separated hosting from management. Customers could keep the installation on their infrastructure while Gitpod managed the service. To make that supportable, Gitpod reduced deployment variance by committing to AWS as the infrastructure provider. The vendor could take back operational responsibility because it no longer had to accommodate every customer’s preferred platform configuration.
That arrangement established a deliberate information boundary:
| Remained with the customer | Returned to Gitpod |
|---|---|
| Development workloads | Operational telemetry |
| Source code and data | Information needed to manage the service |
| Integrations |
The workload and its sensitive assets stayed on customer infrastructure, while a small amount of operational information flowed to the vendor. That division met the requirements of the regulated customers Bichard describes.
Operationally, the model was a success, but it preserved the complexity of the underlying Kubernetes architecture. Installation remained difficult, and multiple clusters imposed a high fixed cost. Reducing the number of supported configurations had made the system easier to manage; it had not removed the infrastructure that made it expensive and complicated in the first place.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Replace the clusters with a small runner
Gitpod ultimately moved away from Kubernetes for this use case. Its engineering account, We’re leaving Kubernetes, develops the workload-specific reasoning behind that decision. The design question became more direct: what is the smallest architecture that can run securely inside regulated companies without imposing a large operational burden on them?
The replacement uses a runner, analogous to the runner an engineer might know from GitHub CI. It keeps source code and access to customer data on the customer’s infrastructure. In the AWS architecture described in the recording, the runner is a single ECS task running one container. Bichard puts the runner’s cost at single-digit dollars per month, separate from the development workloads it launches. This is the historical design described in the talk, not a complete specification for a current deployment.
The runner launches development environments using the capabilities of its host platform. On AWS, those workloads run on EC2 and are backed up to EBS. The same principle applies to the Linux runner: inherit the qualities of the platform on which it runs. Gitpod chose provider-native functionality over Kubernetes’ theoretical portability, avoiding the additional abstraction and operational complexity that portability had brought to its workload.
The remaining question was what Gitpod could operate centrally so customers would not have to. The vendor retained coordination metadata, including IDs and user-related identifiers, while the source code and valuable intellectual property stayed inside customer infrastructure. Bichard characterizes that metadata as non-personally-identifying and less sensitive to these customers than their source code; that is a description of the intended data boundary, not a general guarantee that any user identifier is harmless.
| Component or data | Location in the described architecture |
|---|---|
| Management metadata and user IDs | Gitpod’s side |
| Runner | Customer infrastructure; ECS on AWS |
| Development environments | Customer infrastructure; EC2 on AWS |
| Workload backups | EBS on AWS |
| Source code and data access | Customer infrastructure |
The customer keeps the sensitive workload; the vendor keeps much of the coordination burden. This is also the foundation on which Gitpod runs its agent fleet.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Installation is simple once the network is ready
The deployment video makes the boundary visible in the user experience. A user selects a workload, but execution happens inside the customer’s network and cloud account. The runner setup follows a short sequence:
- Use the runner interface to create a CloudFormation configuration.
- Supply the customer’s network details on the customer’s side.
- Create the installation in that account.
The interface makes provisioning approachable without moving the workload out of the customer’s infrastructure.
Bichard says runner installation can take as little as three minutes once the required network details are ready. The usual bottleneck is obtaining the VPC and subnet information, identifying who owns those resources, and finding someone authorized to allocate them. The claimed installation time therefore describes provisioning, not the entire enterprise approval and onboarding process. A simpler deployment reduces technical overhead, but it does not eliminate the organization’s responsibility for its network.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give agents the environment developers already use
The agent offering grew out of work with customers and design partners dating back to the start of the year. Because Gitpod already had infrastructure inside those customers’ accounts, it could reuse the same development workloads. An autonomous coding agent needs much of what makes a human developer productive: source code, a place to execute and iterate, and access to the internal systems involved in the task.
Those internal systems might include a database or a cluster. Within the existing development environment, an agent can work on source code with the access and privileges available to an individual developer. The architecture provides an execution location and an access boundary for the agent rather than requiring a separate external workspace with a new path into customer systems.
That reuse underpins the offering’s privacy-first positioning. The agent runs inside customer infrastructure, building on the same deployment model developed for regulated organizations. The product panel presents Ona as the software engineering agent hosted in the customer’s VPC.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make platform activity auditable
The redesign also gave Gitpod an opportunity to build the platform around an API-first interface. Bichard says platform actions and interactions—including creating a development environment—are audit logged, giving customers a history of activity on the platform. That audit history extends to agent tasks when they use the same infrastructure. The scope here is platform operations and agent-task activity, rather than a claim that every shell command or file access inside an environment is recorded.
For buyers of AI tools, the underlying architecture therefore belongs in the purchasing decision: where work executes, which infrastructure the customer owns, and what operational qualities come with that arrangement. For vendors, Gitpod’s experience ties technical simplification directly to the customer’s ability to adopt and operate the product. In this case, moving away from a complicated Kubernetes deployment made it possible to preserve the customer’s security boundary while reducing the burden of running the service.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Gitpod’s engineering account of the security, storage and resource-management difficulties behind its departure from Kubernetes.
Further reading
A guide to specifying agent tasks and using isolated development environments for parallel work.
Updates since the talk
Current runner cost categories, exclusions and instructions for separating runner and environment spending in AWS Cost Explorer.
Current VPC connectivity, AWS service endpoint and subnet requirements for deploying runners.
Current audit-event structure, access requirements and ways to query platform activity.
Full transcript
Loading transcript…