Zero Cold Starts: Serverless AI Agents on Akamai Functions — Thorsten Hans
Loading slides…
Read the talk
Zero Cold Starts: Serverless AI Agents on Akamai Functions
Thorsten Hans builds a WebAssembly MCP server, deploys a persistent decision log across service regions, and runs a tool-using agent. The walkthrough explains component composition, deployment health checks and JavaScript memory snapshots—and separates fast startup from the time spent calling a model.
From a talk by Thorsten Hans
At a glance
Ideas worth remembering
“Zero cold starts” describes negligible claimed startup overhead: Akamai Functions still starts the application per request, with Hans reporting an average below half a millisecond.
WasmCP separates tool implementation from server assembly. A compiled tools component becomes an MCP server through composition with the required supporting components.
The decision-log example connects intent, approval, a remote tool call and persistence: an IDE request inserts a decision that a later list operation can retrieve.
JavaScript memory snapshots move runtime and dependency initialization into the build. The artifact must also carry the JavaScript runtime, given as roughly ten megabytes.
A globally deployed agent can still call a model in one location. The successful coin-toss demo reports roughly 820–866 milliseconds for the model loop, separate from application startup.
A global audience exposes two different delays
An MCP server or AI agent still has the basic obligations of an application: receive a request, do useful work and respond quickly. Thorsten Hans, a senior developer advocate at Akamai, opens with two infrastructure problems that can slow that exchange. A deployment in one region leaves some users farther from the application. Separately, scaling containers or virtual machines can require starting new instances before they can handle requests.
Keeping spare instances running can absorb sudden demand, but it makes idle capacity part of the cloud bill. Hans also points to its environmental cost. Higher-level platforms can remove some infrastructure work, yet introduce their own SDKs, packaging formats and production conventions. Once an application depends on those choices, moving it to another provider becomes harder.
Server-side WebAssembly is the proposed alternative deployment unit: a small, portable binary that is cheap to distribute and quick to start. This addresses startup and packaging; distributing those binaries across regions addresses where requests run. The talk develops both parts through Spin and Akamai Functions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Spin builds the application; Akamai Functions runs it
Spin is the vendor-agnostic CNCF project used throughout the demos. Its three pieces serve different parts of development:
- CLI: Build and run applications during the local development loop.
- Runtime: Execute WebAssembly using Wasmtime, including locally and in constrained environments.
- Language SDKs: Give application code convenient ways to call HTTP services, interact with databases and perform other common tasks.
Language-specific SDKs sit above a shared target: WebAssembly with the WebAssembly System Interface, or WASI.
Akamai Functions supplies the managed, globally distributed runtime. The title’s “zero cold starts” needs a precise reading: Hans says an application is actually cold-started for every request, with average startup below half a millisecond. That is a platform claim about startup overhead, rather than a claim that the entire request—including a model call—finishes in that time.
Two supporting mechanisms make the runtime useful beyond stateless computation. WebAssembly’s sandbox requires developers to grant access to external resources explicitly. A built-in, multi-tenant key-value store supplies persistence and is associated with applications that use it, without application developers arranging separate credentials. The decision-log demo later puts that storage path to work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A tools component becomes an MCP server through composition
Implementing MCP directly would mean handling its API over HTTP as well as the useful operations exposed to clients. WasmCP separates those concerns. Its CLI scaffolds tools, resources and prompts, then combines application components with ready-made server components. Hans also describes OAuth2 and OIDC support; the local demonstration deliberately omits authentication, so its successful tool call demonstrates composition rather than an authenticated deployment.
The first project is a TypeScript tools template called Hello. WasmCP also offers Rust and Python templates. After installing dependencies, the application work has three concrete parts:
- Describe inputs: Define the schema of arguments a tool accepts.
- Describe tools: List tools with precise descriptions so people and language models can choose and use them.
- Dispatch calls: Implement
callTool, which examines the request and sends it to the appropriate handler.
The example handler validates incoming arguments, combines the input with a prefix and returns a result.
The build has an important intermediate state. Running make produces a WebAssembly tools component, which is not yet a complete MCP server. wasmcp compose brings in the other required components and produces the server artifact. Spin then runs server.wasm locally on port 3000. In MCP Inspector, Hans connects, lists the example tool and submits foo as its input. The small test checks that the client can discover and invoke the tool through the composed server.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Deploy a persistent decision log only after every region is ready
The next application gives the workflow something worth persisting: a Rust decision-log MCP server. Its structure matches the TypeScript example. callTool dispatches to tools for creating, reading, updating and deleting decisions, and the tool implementations use a store. Locally, Spin provides that key-value store through SQLite. On Akamai Functions, the application uses the platform’s globally distributed key-value store. The demo establishes this change of backing store, but does not explain its consistency or concurrent-update semantics.
The application manifest combines compilation and composition into the Spin build workflow. That keeps the developer from manually repeating the separate steps used for Hello. Hans deploys with spin aka deploy, names the application ADR MCP and authorizes registry access. The command packages the WebAssembly artifact and application manifest into OCI layers and uploads them.
What has to happen before a globally distributed server gets a public address? The deployment flow below makes the readiness condition visible. Each service region must receive the workload, confirm it can run it, instantiate it and pass an internal health check. Only after every region returns HTTP 200 does the platform issue a public subdomain. Hans reports roughly 60 seconds for this deployment.
The public endpoint is also a deployment decision. Hans notes that another service, such as Property Manager, can enforce controls before forwarding traffic to the MCP server. Component-level authentication and controls in front of an endpoint are therefore separate choices developers still need to make, even when packaging and regional rollout are managed.
Regional readiness gates the public endpoint
Read the diagram as text
- Package the application. Create OCI layers from the WebAssembly file and application manifest.
- Distribute to all service regions. Each region confirms receipt and its ability to run the workload.
- Instantiate and check. Run an internal health check in every region.
- Issue the public subdomain. Publish the endpoint after all regional checks return HTTP 200.
- Package the application → Distribute to all service regions: Upload and distribute.
- Distribute to all service regions → Instantiate and check: Confirm the workload can run.
- Instantiate and check → Issue the public subdomain: Every region returns HTTP 200.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An IDE request becomes a stored decision
The decision log now changes from deployed infrastructure into a usable project tool. Hans adds the remote endpoint to Zed, which discovers five tools. In the project’s chat, he asks it to record a decision about using OAuth2 with Auth0. The client identifies the insert-decision operation and presents the raw tool input for approval. Once approved, the call reaches Akamai Functions and the decision is stored.
A follow-up request—“Give me all the decisions, please”—selects the list-decisions tool. The observable change is that a decision introduced through conversation can now be retrieved through another tool call. Tool descriptions help the client select an operation; the approval step lets the user inspect the proposed write; the server’s storage gives the next request something to read. Akamai Functions routes calls to the closest service region, while the MCP interface lets clients use the server without depending on the language in which its tools were written.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run the agent at the edge, and prepare JavaScript before requests arrive
The agent demo keeps the same deployment workflow but changes the application code. A TypeScript Spin app uses the Vercel AI SDK and calls an Ollama instance on a Linode server with a dedicated GPU. Its tools can flip a coin or roll a die with a requested number of sides. The HTTP handler loads configuration, selects the model endpoint, gives the agent an identity and instructions, registers its tools, limits the number of steps and returns the generated response.
This topology matters: deploying the agent globally does not deploy the model globally. The request handler and tool orchestration run on Akamai Functions; model inference remains on the GPU-backed Linode instance. A fast-starting agent still has to reach that model and wait for its work.
While the game agent uploads, Hans explains how interpreted JavaScript becomes a WebAssembly artifact. At build time, the toolchain loads the interpreter and application code, then evaluates the global scope. With the runtime initialized and dependencies loaded, it takes a memory snapshot and writes that state into the WebAssembly artifact. Initialization work has moved earlier in the application’s lifecycle.
Which work moves out of request startup, and what must travel with the application? The diagram follows the snapshot’s contents into the deployable artifact. The benefit is already-initialized runtime and dependency state; the cost is carrying the JavaScript runtime with the user code. Hans gives its size as roughly ten megabytes. This mechanism explains the startup benefit, but does not establish his broader claim that every JavaScript WebAssembly application will also execute faster than native JavaScript.
JavaScript initialization becomes part of the artifact
Read the diagram as text
- Load interpreter and code. Build time — Prepare the JavaScript runtime and custom application code during compilation.
- Evaluate global scope. Build time — Initialize the runtime and load dependencies before deployment.
- Capture memory. Build time — Take a snapshot of the initialized state.
- Write the WebAssembly artifact. Deployable output — Store the snapshot with the application; include the roughly ten-megabyte JavaScript runtime.
- Load interpreter and code → Evaluate global scope: Evaluate the loaded application.
- Evaluate global scope → Capture memory: Capture initialized runtime and dependencies.
- Capture memory → Write the WebAssembly artifact: Write memory state to Wasm.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A fast startup still leaves a model round trip
The deployed game agent receives “Toss a coin.” Response headers identify Santa Clara as the entry point into Akamai’s network and LAX as the closest service region. Hans initially reads a value of about 34 milliseconds from x-envoy-upstream-service-time as the complete interaction, then notices an internal server error. That failed request cannot stand as the successful agent timing.
On retry, the reported time is 866 milliseconds and the coin result is tails. Further calls eventually produce heads, with Hans reporting roughly 820 milliseconds for a full loop through the model. These are demonstration timings rather than a latency distribution or an independently established end-to-end measurement. They still expose the useful distinction: removing substantial startup overhead leaves model inference and communication as real parts of an agent request.
The ending returns to a common developer loop: implement in a language targeting WebAssembly, test locally with Spin, deploy with spin aka deploy, then point clients at the endpoint. Rust, TypeScript, Python and Go are the examples named. MCP servers add component composition; agents bring their SDK, model configuration and tools. Both remain applications built and deployed through the same framework.
Portability is the closing reason to keep the application in WebAssembly and WASI rather than tie its packaging to one cloud. Hans presents Akamai Functions as one runtime for Spin applications, with global distribution as its particular offering. Moving the executable is only part of moving the application: the decision log also uses platform-provided persistence, and the game agent uses an external model endpoint.
The operational promise is less infrastructure to assemble and maintain. It does not end at the deploy command: Hans points to commands for metrics, logs and upgrades, and closes by directing developers to the Akamai Developer Hub for further experiments and experiences. The workflow gives application developers a shorter path from local tools to a remote service while retaining ways to inspect and update what they have shipped.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Full transcript
Loading transcript…