AI Engineer World's Fair 2026
Move Fast and Don't Break Things: Scaling Databases for the AI Era — PlanetScale
Loading slides…
Read the talk
Move Fast and Don't Break Things: Scaling Databases for the AI Era
Ben Dicken explains how isolation, redundancy and overload controls support growing databases—and how configuration files, resource budgets and schema branches give AI agents safer ways to operate them.
From a talk by Ben Dicken
At a glance
Ideas worth remembering
Isolation keeps a failed supporting component from taking down the data plane; redundancy prepares each database shard to survive server failure.
Sharding distributes data and queries, backpressure limits overload, and decoupling lets workloads scale independently.
A JSON VSchema gives an agent an editable sharding plan while Vitess and VTGate handle query routing underneath.
Traffic Control applies resource budgets to tagged traffic, with warnings or query termination to preserve capacity under excess demand.
Schema branches, deploy requests and reverts give agents the same controlled change workflow built for human developers, with human review where desired.
Agent speed meets infrastructure limits
Shipping faster creates value only while the application can keep serving its users. AI agents intensify both sides of that problem: they help write and deploy features, and they generate demand that existing infrastructure must absorb. In this talk, Ben Dicken of PlanetScale starts with the familiar ambition to “move fast” and asks how to keep databases and application servers healthy while doing it.
Rapid growth makes the tension concrete. An application can gain multiples of its previous daily active users while its infrastructure struggles to catch up. The desired result is a healthy status page and three, four or perhaps five nines of availability. Those are aspirations here, rather than measured results for a demonstrated system. Scaling to tens or hundreds of millions of users remains a hard engineering problem.
PlanetScale operates databases for fast-growing AI companies, including Cursor. The explanation proceeds through three questions: how to keep systems available at any size, how to increase their capacity as usage grows, and how to give agents some control over that infrastructure without making each change a company-wide gamble.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Contain failures, then prepare for them
Isolation begins with a specific failure: a bad deployment takes down the control plane. The data plane—the databases holding the critical information users need to log in and use the application—should remain accessible. Observability pipelines, analytics systems, apps and MCP components can also fail. Keeping them isolated prevents a frequently changed supporting component from bringing the database down with it.
Redundancy addresses a different problem: the database server itself can fail. Replacing stateless application compute is comparatively straightforward; another instance can handle requests without having to recover the application's stored data. Database failover must preserve state. The common arrangement described here sends most traffic to a primary and keeps replicas or followers in other data centers or availability zones ready to take over. That adds cost and complexity. The talk does not specify replication or failover protocols, so the topology alone does not establish a zero-data-loss guarantee.
Fleet size changes how often the operator encounters failure. With one server and 100 users, a failure might be a once-in-several-years event. Across thousands of servers, failures can become weekly or monthly events even when individual machines are reasonably dependable. Reliability therefore requires a routine way to survive broken machines. Isolation and redundancy remain useful from small applications through millions of daily active users; growth adds capacity problems rather than retiring those principles.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reject excess work and scale services independently
More servers do not eliminate overload. Too many queries, excessive CPU use or exhausted cache memory can still bring a database down. Backpressure makes the system respond before that failure: once resource use crosses a threshold, it can fail requests, deny queries or refuse connections. Some traffic degrades so that the remaining traffic can keep working. The rest of the application must support those responses; a database's refusal to accept work needs to be an expected operating condition.
Decoupling addresses the resource competition that appears when several jobs share one system. Combining transactional database work, analytics, queues, job processing and caching has an obvious convenience: fewer services to operate. But Dicken identifies two costs:
- Shared failures: When the combined service goes down, all of those functions can go down together, undermining isolation.
- Shared resources: A burst of queued jobs can compete with other workloads even when those workloads have not grown.
The queue burst gives decoupling a practical test. If only the queue needs more compute, separating it lets the operator increase its resources independently of the transactional database, analytics or cache. Sharding, backpressure and decoupling solve distinct parts of scaling: distributing work, limiting admitted work and separating workloads whose capacity needs can diverge. They are useful building blocks, rather than a complete account of database scalability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give the agent a sharding configuration it can work on
Generating application code and changing a production database carry different consequences. A mistaken configuration or sharding change can affect the entire application, and recovery may be much harder than undoing a code edit. The next question is therefore how an agent can work through a manageable interface while the database platform handles the complicated infrastructure underneath.
Vitess provides a concrete example. Its MySQL shards sit behind VTGate, an intelligent proxy that parses queries, determines which shard they need and routes them. The sharding design is expressed through a VSchema: a JSON file that specifies the column to shard on, the sharding method and table-placement choices. The important distinction is between a complicated execution system and an editable configuration describing how it should distribute data and queries.
json
{
"sharded": true,
"vindexes": {
"xxhash": {
"type": "xxhash"
}
},
"tables": {
"user": {
"column_vindexes": [
{
"column": "user_id",
"name": "xxhash"
}
]
}
}
}
The proposed agent workflow starts with an existing schema and a scalability problem. Give those inputs to the agent, ask it to prototype a VSchema, then develop the sharding design before deploying the configuration through PlanetScale's interfaces. APIs and a command-line interface allow the agent to assist with, or take some control over, that process. Text and JSON make the work accessible to an agent; they do not by themselves prove that its sharding choices are efficient. No particular schema, generated plan or benchmark is developed in the spoken example.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Budget traffic before an agent surge exhausts the database
Agent-generated demand brings the earlier backpressure principle into focus. Dicken recalls GitHub reporting a large traffic increase attributed to agents, although his recollection leaves the multiplier uncertain. PlanetScale's Traffic Control addresses the operating problem by categorizing and tagging database traffic, then assigning resource budgets to those categories. A segment of traffic can be limited by CPU usage or backend processes instead of being allowed to consume whatever the database has left.
What does exceeding the budget look like? The graph described in the talk shows the amount of queries exceeding a configured budget over time. Traffic Control can then respond at different levels:
- Warnings: Send feedback to the client that the database is overloaded and it needs to slow down.
- Stricter enforcement: Kill executing queries to enforce the limit.
The graph makes budget exceedance visible; the response determines whether the system merely warns about excess work or actively stops it.
Rejecting work has a cost: some requests fail. The intended benefit is to preserve service for other traffic rather than let overload take the entire application offline. This also closes the feedback loop introduced earlier: warnings only help if clients respond by slowing down, while enforced limits can protect capacity even when incoming demand keeps rising.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Mirror code branches with isolated schema changes
Application features often require schema changes alongside code changes: new tables, new columns, deletions or indexes. Developers already have a familiar way to isolate code work—branch from main, edit, review, merge and deploy. PlanetScale applies that Git-like experience to the database schema. A schema branch provides an isolated environment for changes; a deploy request merges those changes into production, with no downtime as described in the talk.
What happens if the deployed schema causes a problem? The interface includes a one-click revert, which Dicken describes as returning to the previous database state without data loss. In this schema-change context, the useful promise is recovery from the deployment without sacrificing data. The talk does not explain the underlying revert mechanism or its operating constraints, so this should not be extended into a guarantee that any database operation can be undone.
APIs and CLIs make the same workflow available to agents. An agent creating code branches, pull requests and reviews can also maintain a corresponding database schema state, keeping the two kinds of changes in sync. Automation can be full or partial; human review remains an explicit option. Isolation, deployment and recovery are platform primitives the workflow can use rather than procedures the agent must invent each time.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Operational developer experience becomes agent infrastructure
The ending gives developer experience an operational definition. Light and dark modes or keyboard shortcuts may improve an interface, but the consequential tools help the person responsible for a database deploy safely, avoid mistakes and improve performance. These capabilities existed for human developers before agents; agents now benefit from the same deliberate ways to configure, isolate and recover changes.
PlanetScale exposes many of those capabilities through an MCP server and command-line interfaces. Dicken leaves the debate over which interface is better open. The useful conclusion is more concrete: when infrastructure gives operators clear configuration inputs, resource controls and safe change workflows, it also gives agents practical tools for helping databases scale. The quality of the underlying operations matters as much as the agent's ability to call them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Full transcript
Loading transcript…