AI Engineer World's Fair 2025
How we hacked YC Spring 2025 batch’s AI agents
Loading slides…
Read the talk
How we hacked YC Spring 2025 batch’s AI agents
Three attacks on YC startups trace how agent tools can expose other users’ records, undermine a code sandbox, and send private repository credentials to an attacker.
From a talk by Rene Brandel
Before you start: Basic familiarity with API requests, authentication tokens, and containers will help you follow the attack paths.
A blog built by voice, across three clouds
Ask a computer to build a blog post, then have it generate the site and upload pictures from San Francisco. Rene Brandel was building that workflow roughly a decade before this talk, using voice commands and APIs that made the demonstration visibly slow. He reports that the voice-to-code project won Europe’s largest hackathon. Brandel introduces himself as CEO of Casco, a YC company focused on red teaming AI agents and applications, after previously working on agents at AWS.
The old project worked, but assembling it required substantial plumbing before modern generative AI made this kind of interface straightforward. Two months before the talk, Brandel had left AWS to work with his co-founder in a garage; they subsequently joined Y Combinator. Looking back at his hackathon architecture, he shows three cloud providers, including IBM Watson and Microsoft LUIS. The diagram connects a React application to speech recognition and several language services: much of the engineering was making separate components cooperate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Every connection is part of the security boundary
The contemporary stack is more recognizable: a front end talks to an API server, the server talks to an LLM, and the model works through tools connected to data sources. That consistency makes applications easier to build. It also gives security reviews a concrete structure: examine each connection and the authority that crosses it.
Agent security extends beyond model behavior. Prompt injection and harmful content matter, but neither captures everything a tool can do with a database credential, a writable filesystem, or access to an internal network. The consequential failure may sit several connections downstream of the LLM. Brandel’s focus is the arrows in the architecture diagram: where requests travel, which permissions accompany them, and what those permissions allow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Thirty minutes per agent
The exercise began with a marketing objective: produce a memorable internal Y Combinator launch. Brandel says the resulting post became YC’s second-highest-upvoted internal launch, ahead of Rippling. To find material for it, the team selected agents that were already live and imposed a short testing budget.
Their sequence was straightforward:
- Set a thirty-minute timer for an agent.
- Try to extract its system prompt to understand its intended behavior.
- Inspect tool definitions, especially capabilities that access data or execute code.
- Attempt to turn those capabilities into unauthorized actions.
Brandel reports compromising seven of sixteen launched agents within a thirty-minute testing limit per agent. This was a bounded exercise on that group of applications; the talk does not specify a standardized compromise criterion. The findings organize the rest of the demonstration into three recurring issues.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A valid token does not authorize every record
The first issue was cross-user data access. Brandel introduces the examples as having been remediated, joking that investors could now consider the companies secure. In this case, an extracted system prompt exposed tools for looking up user information and documents by ID. Those interfaces suggested a familiar vulnerability: Insecure Direct Object Reference, or IDOR. The application could validate a token yet fail to check whether its holder was allowed to access the requested object.
An opaque ID does not fix that missing check. The team found a user ID in the URL bar of a publicly recorded product demonstration and supplied it to the lookup tool. Brandel reports receiving personal information, including an email address and nickname. The ID displayed during his talk was a replacement belonging to his co-founder.
The lookup also exposed relationships. A user record led to a chat ID and a document ID; those identifiers could become inputs to further tools. A single unauthorized lookup therefore provided routes through connected records. The failure was not simply that the model revealed one answer—it was that the backend accepted identifiers without enforcing the caller’s access rights.
The fix requires two distinct checks:
| Check | Question |
|---|---|
| Authentication | Is this token valid, and who is calling? |
| Authorization | May this caller access this particular object? |
Brandel points to access-control rules and Supabase row-level security as ways to enforce the second check. RLS must run under an appropriately restricted role; an administrative role that bypasses RLS defeats that protection. The important boundary is downstream, where a tool actually reads the data.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Treat the agent as a user
Why would developers who know how to build a web application omit these checks in an agent? In discussions with YC companies, Brandel encountered a recurring assumption: the agent runs on a server, so it should behave like a service and receive service-level permissions. Hosting location had become a substitute for deciding whose authority the operation should use.
The model can choose an operation; it should not decide whether the caller is authorized to perform it. Treating an agent as a user means keeping that decision in trusted application code and avoiding broad service permissions for user-directed work. A small Python example makes the separation concrete: caller_id comes from verified authentication, while the model supplies only the requested document identifier.
python
from dataclasses import dataclass
@dataclass(frozen=True)
class Document:
id: str
owner_id: str
text: str
DOCUMENTS = {
"doc-17": Document("doc-17", "user-b", "Private planning notes")
}
def read_document(*, caller_id: str, document_id: str) -> str:
document = DOCUMENTS.get(document_id)
if document is None or document.owner_id != caller_id:
raise PermissionError("Document unavailable")
return document.text
With authenticated caller user-a and requested document doc-17, this function denies access. Knowing another user’s document ID does not grant permission to read it. This owner-only example illustrates the boundary; applications with sharing would enforce their access-control rules at the same point.
The same treatment extends to tool inputs and outputs. Validate and sanitize what enters an operation, and handle its output safely at the destination. Brandel connects this to the preceding OAuth talk: the security practices developed for users and web applications still apply when an LLM chooses the next request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
When the sandbox can rewrite its own protections
The second issue involves code execution, which Brandel describes as less common but potentially more damaging. He introduces it with an Anthropic occupational-usage comparison for Claude. The matching Anthropic Economic Index report compares computer and mathematical occupations’ share of U.S. workers with their associated tasks’ share of sampled Claude.ai conversations—not programmers’ share of the world population or all Claude usage.
Code tools are useful beyond coding assistants. An agent might generate a calculator on demand to answer a numerical question. But the tool-to-container connection gives the application access to arbitrary compute, making the execution environment a critical security boundary.
The team again began with prompt extraction. The prompt itself was reconnaissance: it revealed what the developer wanted to prevent. Instructions to keep generated code hidden suggested asking to see it; instructions to execute at most once suggested trying repeated execution. The goal was to discover which restrictions were merely instructions and which were enforced by the runtime.
Initially, the environment appeared constrained. It accepted Python, blocked dangerous function calls, and restricted which Python files could run. Two remaining capabilities nevertheless mattered: it could write a Python file and read files. The team asked for filesystem-tree functionality and used the resulting listing to discover app.py.
Reading app.py exposed the service’s write-file and execute-file endpoints. Those endpoints were behind a VPC and could not be reached directly from outside, but the file also contained the code-execution security checks. The existing write capability could modify that same file. Brandel reports overwriting the checks with empty strings, removing the protections that were supposed to constrain execution. His Bitcoin-mining remark illustrates possible compute abuse, rather than a reported mining operation. The decisive mistake was allowing untrusted work to modify the code enforcing its restrictions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From code execution to BigQuery
Control of the container opened another path: discovering the surrounding infrastructure. Brandel describes using service endpoint and metadata discovery to identify network resources, then retrieving a service token, the project name, and the token’s scopes. The token’s permissions determined how far the compromise could extend beyond the execution environment.
Those permissions were too broad. Brandel reports that the team queried BigQuery and gained access to all of the affected company’s customer data. The full sequence runs from writable execution controls to arbitrary code, then through cloud metadata and service credentials to a data service. A container compromise became a customer-data breach because the surrounding identity and network boundaries allowed lateral movement.
Brandel’s recommendation is analogous to avoiding homemade authentication: use an established code sandbox instead of building the execution boundary as a small Python service. He names E2B and praises an unnamed company in his YC batch for built-in observability, quick startup, and an MCP server that makes it easy for agents to connect. These are reasons to evaluate a sandbox service, not evidence that a particular integration has been configured securely.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A repository string redirects credentials
The third issue was server-side request forgery, or SSRF: inducing a service to contact an endpoint it was not intended to contact. Here, the extracted system prompt revealed a database-creation tool that fetched a schema from a private GitHub repository. Private access implied that the fetch had credentials available to it.
The repository location was supplied as a string. The team substituted an attacker-controlled repository address and observed the request arriving there with Git credentials. Brandel reports that those credentials then enabled downloading the private codebase. The dangerous combination was control over the outbound destination plus credentials that followed the request to that destination.
Brandel says the team immediately notified the affected batch mates, who reported that they had fixed the issue. The broader implementation lesson is to inspect what each tool actually does with its inputs: a repository parameter can determine both where a server connects and where privileged information travels. Fast development, including vibe coding, does not remove the need to inspect those connections and apply established input and output validation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Testing beyond the prompt
Together, the cases make the security review larger than an LLM evaluation. User-scoped authorization must survive tool calls, and untrusted code must remain contained even when the model ignores instructions. Brandel warns that a sandbox which starts as an intern-sized project can become an infrastructure problem. He presents these vulnerabilities as basic examples, not an exhaustive inventory, and closes the prepared talk with Casco’s approach: an AI agent that attacks other agents and reports where they break, available through a demo.
The first audience question returns to the starting point of each investigation: how did the team obtain system prompts? Brandel points to publicly available techniques and recommends HiddenLayer’s work. The relevant published resource is Policy Puppetry, which addresses system-prompt leakage. He does not provide an extraction recipe in the Q&A.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Approvals and isolation solve different problems
A second audience question challenges command allowlisting: coding agents can find creative ways around permitted commands, so how can execution be controlled? The question becomes especially consequential locally, where the agent may have access to the credentials of the user running it.
Brandel distinguishes the local approval decision from server-side containment:
- Local execution: He characterizes the choices at the time as unrestricted execution or asking permission every time, referring to Cursor’s then-used YOLO terminology.
- Server execution: He recommends a code sandbox that constrains access to internal networks and limits how long the environment can live.
An approval policy governs when execution is allowed. A sandbox constrains what the resulting process can reach and how long it can persist.
The final follow-up asks whether those sandboxes use virtual machines. Brandel says they typically use Firecracker underneath and ends with a forceful warning against relying on containers for isolation. The technical distinction needs precision: containers do provide isolation mechanisms, including namespaces, but shared-kernel risks, capabilities, and mounts can weaken that boundary. Firecracker adds hardware virtualization through KVM microVMs, while still depending on correct host configuration. For untrusted generated code, placing a process in a container does not by itself establish a sufficient execution boundary.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
HiddenLayer's research on Policy Puppetry, safety bypasses and system-prompt leakage.
Current guidance for database access policies and the administrative roles that can bypass them.
Current quickstarts and documentation for E2B sandboxes, SDKs, networking and commands.
KVM-based microVM technology with architecture, setup and security guidance for isolated workloads.
Casco's security-testing service and demo request destination.
Further reading
The February 2025 analysis of Claude usage by occupational task, including the computer-and-mathematical usage and workforce comparison.
Explains container isolation mechanisms, capability restrictions and configuration risks.
Full transcript
Loading transcript…