AI Engineer World's Fair 2025
Design like Karpathy is watching 😎
Loading slides…
Read the talk
Design like Karpathy is watching
MenuGen turned restaurant menus into food illustrations, but getting it deployed exposed a harder problem: making APIs, documentation, and onboarding work for developers using AI assistants.
From a talk by Zeke Sikelianos
Before you start: Basic familiarity with HTTP APIs, JSON, and AI coding assistants will help; OpenAPI and MCP are explained as they appear.
Put current instructions into the assistant’s context
The first repair is to make documentation easy to consume as text. Sikelianos points to llms.txt and Markdown versions of documentation as ways to supply useful material without the surrounding HTML interface. The linked proposal has since evolved; the approach demonstrated here is straightforward text delivery. Karpathy’s preference, as quoted in the talk, is a single Markdown document and a copy button over elaborate documentation-page presentation.
Replicate responded by adding several routes from a model page into an assistant’s context:
- Copy Markdown: take the page’s instructions into the tool where the work is happening.
- Open in Claude: start a conversation with Claude about the model and its capabilities.
- Open in ChatGPT: use ChatGPT to explore the model interactively.
These routes turn documentation from a page the user must search into material an assistant can discuss with them.
The same Markdown also serves implementation. A developer using Cursor or Windsurf can put the model’s documentation into the editor so the assistant has current instructions for running it. This addresses the stale-knowledge problem directly: the relevant usage information enters the current conversation instead of depending entirely on what the model learned during training.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the operational contract easy to read
Text documentation becomes especially useful when it includes a complete request. Sikelianos quotes Karpathy: “LLMs don't like to click, they like to curl.” The appeal of curl is its familiarity and independence from a specialized client interface. A request may look syntactically busy, but it puts the details needed to call a service in one place.
| Request detail | What the example communicates |
|---|---|
| Endpoint and HTTP method | Where to send the request and which operation to invoke |
| JSON payload | Which inputs the operation expects |
| Credentials | How to authenticate |
| Response type | What representation to request |
| Blocking or asynchronous behavior | Whether to wait or handle completion separately |
Together, these details make an example a compact operational contract. An assistant can use it to construct a request without reconstructing the interface from scattered descriptions.
Cog applies the same idea to model development. It packages machine-learning models in Docker containers and gives them standardized inputs and outputs described through OpenAPI. Replicate collected Cog’s documentation into a single llms.txt file so an editor could bring the relevant instructions into context.
The workflow starts with an unfamiliar project:
- Clone an existing open-source Cog model.
- Open the project in an AI-assisted editor.
- Reference Cog’s
llms.txtdocumentation. - Ask the editor to use that context while changing the model’s code.
The documentation supplies framework knowledge alongside the project being edited. This leads to Sikelianos’s central design thesis: treat the LLM as a primary audience for your product, service, or library. The human still has the goal, but an assistant increasingly reads the instructions and performs the integration work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use one API description for documentation and tools
To explain MCP, Sikelianos first establishes what OpenAPI contributes. An OpenAPI document describes an HTTP API in JSON or YAML. It includes paths, operations, query parameters, request bodies, and responses; it is more than a JSON Schema for an individual payload. For Replicate, that description covers tasks such as creating predictions, retrieving predictions, and searching for models.
Replicate’s human-facing HTTP API reference is generated from that description. A template renders the structured definition into a page a developer can read. The public-model search example shows the result: documentation organized around an endpoint, its description, and an example request.
The same investment supports assistant-facing tools. Sikelianos introduces Model Context Protocol through Replicate’s approach of turning its OpenAPI description into capabilities an assistant can use. That conversion is an implementation strategy, not the definition of MCP itself: MCP standardizes communication for tools and context through JSON-RPC. OpenAPI describes the HTTP service; the MCP integration makes service operations available to the assistant.
The demonstrated Replicate MCP setup uses Claude Desktop’s developer settings, a JSON configuration, and a Replicate API token. Sikelianos describes the local server setup as handled through that client workflow. This is a historical desktop demonstration; current documentation distinguishes local installation from hosted remote connections, so it should not be read as a universal promise of installation-free setup. Once connected, Claude can issue authenticated Replicate API requests.
That connection supports several related activities:
- Discovery: ask what the service can do before knowing its interface.
- Search: have the assistant find relevant models.
- Project scaffolding: use those capabilities while assembling the beginning of an application.
The foundation is a well-documented, comprehensive OpenAPI description. Sikelianos emphasizes that this existing investment made exposing Replicate’s capabilities through MCP much easier. He names Claude, GitHub Copilot, Visual Studio Code, and the OpenAI Agents SDK as integration contexts in the expanding tool ecosystem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let legitimate users get to work
Documentation and tool access do not help if the service blocks a legitimate customer. Returning to MenuGen, Sikelianos describes Karpathy registering for an API key, entering credit-card details, and then generating many images of French toast. In Sikelianos’s account, an approximately one-hour-old account sending 1,000 requests triggered an abuse mechanism and was blocked. Those numbers describe the incident as he recounts it, not a published rate-limit threshold.
Replicate’s CEO saw the public criticism and intervened to unblock the account, according to Sikelianos. That resolved one user’s problem, but most users cannot rely on a widely read blog post to get access restored. A product needs an ordinary path for legitimate users who arrive ready to do substantial work.
Sikelianos proposes prepaid credit as one such path: a user should be able to commit money up front and then use the service intensively. His $500 example illustrates that intent; it is not an announced price or a guarantee of unrestricted usage. He describes the change as work in progress. From there, he turns to another completion criterion: shipping a feature cannot end at merging its pull request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Document features and keep responses useful
Documentation is part of shipping. A feature needs published instructions that users can discover and an assistant can consume and put to work. Producing those instructions in accessible text formats is part of the product, not an optional presentation layer added after implementation.
Technology choices affect this experience too. Sikelianos uses SQL as the example of a mature language with a large body of existing usage. His rationale for choosing established, stable technology is that models have encountered many examples and have fewer changing conventions to navigate. This is a design heuristic about familiarity, not a measured guarantee of correct generated code.
Finally, an API response may become input to a model with a limited context window. Returning every available field about every model can bury the information needed for the task. Sikelianos recommends smaller, information-dense JSON responses: retain useful details while avoiding an exhaustive metadata dump. The response’s usefulness to its next reader matters as much as its completeness as a database export.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From generated documentation to informed decisions
An audience question asks how to get started generating API documentation. Sikelianos’s answer begins before the choice of documentation site: write an OpenAPI description in YAML or JSON that captures the API’s behavior. He names Docusaurus, Read the Docs, and ReadMe as examples in the documentation ecosystem. Schema-driven tooling can also produce SDKs and clients in different languages, though those capabilities are not necessarily native to every named product. The reusable asset is the API description.
The final question extends the discussion from integration to discovery and distribution. If assistants increasingly inform purchasing decisions—and might eventually make some of them—what does a service need to expose? Sikelianos starts with strong search and API access to the information a user needs to make an informed choice.
Pricing is his concrete example. At the time of the talk, Replicate users had to visit pricing pages or individual model pages to find it. Exposing that information as public, structured JSON would let an assistant retrieve costs and compare options inside the conversation. This is a proposed capability in the discussion, not a demonstrated pricing endpoint.
The imagined request is to compare video models such as Veo, Kling, and MiniMax by expense, speed, and output quality. Pricing data would support the cost comparison; speed and quality would require corresponding evidence of their own. No model ranking is demonstrated. The substantive requirement is that the facts needed for a decision be available in a form the assistant can retrieve and reason over. Search can find the options, but structured decision data makes those options possible to evaluate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Karpathy's account of building, deploying, and adding payments to a menu-illustration app.
Text documentation for packaging models in containers and exposing typed inference APIs.
Further reading
- Zeke's companion pageArticle
The speaker's presentation page with official and rough-cut recordings and a transcript.
- MCP specification from March 2025Documentation
A historical specification for connecting applications to model-facing context and tools.
The formal structure for describing HTTP operations, parameters, request bodies, and responses.
Updates since the talk
Instructions for connecting supported assistants to Replicate through local or remote MCP servers.
A later search endpoint covering models, collections, and documentation, with MCP response filtering.
Full transcript
Loading transcript…