AI Engineer World's Fair 2025

Design like Karpathy is watching 😎

Loading slides…

Read the talk

Design like Karpathy is watching

MenuGen turned restaurant menus into food illustrations, but getting it deployed exposed a harder problem: making APIs, documentation, and onboarding work for developers using AI assistants.

From a talk by Zeke Sikelianos

Before you start: Basic familiarity with HTTP APIs, JSON, and AI coding assistants will help; OpenAPI and MCP are explained as they appear.

Put current instructions into the assistant’s context

The first repair is to make documentation easy to consume as text. Sikelianos points to llms.txt and Markdown versions of documentation as ways to supply useful material without the surrounding HTML interface. The linked proposal has since evolved; the approach demonstrated here is straightforward text delivery. Karpathy’s preference, as quoted in the talk, is a single Markdown document and a copy button over elaborate documentation-page presentation.

Replicate responded by adding several routes from a model page into an assistant’s context:

  • Copy Markdown: take the page’s instructions into the tool where the work is happening.
  • Open in Claude: start a conversation with Claude about the model and its capabilities.
  • Open in ChatGPT: use ChatGPT to explore the model interactively.

These routes turn documentation from a page the user must search into material an assistant can discuss with them.

The same Markdown also serves implementation. A developer using Cursor or Windsurf can put the model’s documentation into the editor so the assistant has current instructions for running it. This addresses the stale-knowledge problem directly: the relevant usage information enters the current conversation instead of depending entirely on what the model learned during training.

5:195:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:19 · section reference included

Make the operational contract easy to read

Text documentation becomes especially useful when it includes a complete request. Sikelianos quotes Karpathy: “LLMs don't like to click, they like to curl.” The appeal of curl is its familiarity and independence from a specialized client interface. A request may look syntactically busy, but it puts the details needed to call a service in one place.

Request detailWhat the example communicates
Endpoint and HTTP methodWhere to send the request and which operation to invoke
JSON payloadWhich inputs the operation expects
CredentialsHow to authenticate
Response typeWhat representation to request
Blocking or asynchronous behaviorWhether to wait or handle completion separately

Together, these details make an example a compact operational contract. An assistant can use it to construct a request without reconstructing the interface from scattered descriptions.

Cog applies the same idea to model development. It packages machine-learning models in Docker containers and gives them standardized inputs and outputs described through OpenAPI. Replicate collected Cog’s documentation into a single llms.txt file so an editor could bring the relevant instructions into context.

The workflow starts with an unfamiliar project:

  1. Clone an existing open-source Cog model.
  2. Open the project in an AI-assisted editor.
  3. Reference Cog’s llms.txt documentation.
  4. Ask the editor to use that context while changing the model’s code.

The documentation supplies framework knowledge alongside the project being edited. This leads to Sikelianos’s central design thesis: treat the LLM as a primary audience for your product, service, or library. The human still has the goal, but an assistant increasingly reads the instructions and performs the integration work.

7:037:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:03 · section reference included

Use one API description for documentation and tools

To explain MCP, Sikelianos first establishes what OpenAPI contributes. An OpenAPI document describes an HTTP API in JSON or YAML. It includes paths, operations, query parameters, request bodies, and responses; it is more than a JSON Schema for an individual payload. For Replicate, that description covers tasks such as creating predictions, retrieving predictions, and searching for models.

Replicate’s human-facing HTTP API reference is generated from that description. A template renders the structured definition into a page a developer can read. The public-model search example shows the result: documentation organized around an endpoint, its description, and an example request.

Browser showing the Search public models documentation, with Endpoint, Description and Example cURL request sections on a pink background.Open full source frame
Replicate's API reference for searching public models.

The same investment supports assistant-facing tools. Sikelianos introduces Model Context Protocol through Replicate’s approach of turning its OpenAPI description into capabilities an assistant can use. That conversion is an implementation strategy, not the definition of MCP itself: MCP standardizes communication for tools and context through JSON-RPC. OpenAPI describes the HTTP service; the MCP integration makes service operations available to the assistant.

The demonstrated Replicate MCP setup uses Claude Desktop’s developer settings, a JSON configuration, and a Replicate API token. Sikelianos describes the local server setup as handled through that client workflow. This is a historical desktop demonstration; current documentation distinguishes local installation from hosted remote connections, so it should not be read as a universal promise of installation-free setup. Once connected, Claude can issue authenticated Replicate API requests.

That connection supports several related activities:

  • Discovery: ask what the service can do before knowing its interface.
  • Search: have the assistant find relevant models.
  • Project scaffolding: use those capabilities while assembling the beginning of an application.

The foundation is a well-documented, comprehensive OpenAPI description. Sikelianos emphasizes that this existing investment made exposing Replicate’s capabilities through MCP much easier. He names Claude, GitHub Copilot, Visual Studio Code, and the OpenAI Agents SDK as integration contexts in the expanding tool ecosystem.

9:349:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:34 · section reference included

Let legitimate users get to work

Documentation and tool access do not help if the service blocks a legitimate customer. Returning to MenuGen, Sikelianos describes Karpathy registering for an API key, entering credit-card details, and then generating many images of French toast. In Sikelianos’s account, an approximately one-hour-old account sending 1,000 requests triggered an abuse mechanism and was blocked. Those numbers describe the incident as he recounts it, not a published rate-limit threshold.

Replicate’s CEO saw the public criticism and intervened to unblock the account, according to Sikelianos. That resolved one user’s problem, but most users cannot rely on a widely read blog post to get access restored. A product needs an ordinary path for legitimate users who arrive ready to do substantial work.

Sikelianos proposes prepaid credit as one such path: a user should be able to commit money up front and then use the service intensively. His $500 example illustrates that intent; it is not an announced price or a guarantee of unrestricted usage. He describes the change as work in progress. From there, he turns to another completion criterion: shipping a feature cannot end at merging its pull request.

13:0913:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:09 · section reference included

Document features and keep responses useful

Documentation is part of shipping. A feature needs published instructions that users can discover and an assistant can consume and put to work. Producing those instructions in accessible text formats is part of the product, not an optional presentation layer added after implementation.

Technology choices affect this experience too. Sikelianos uses SQL as the example of a mature language with a large body of existing usage. His rationale for choosing established, stable technology is that models have encountered many examples and have fewer changing conventions to navigate. This is a design heuristic about familiarity, not a measured guarantee of correct generated code.

Finally, an API response may become input to a model with a limited context window. Returning every available field about every model can bury the information needed for the task. Sikelianos recommends smaller, information-dense JSON responses: retain useful details while avoiding an exhaustive metadata dump. The response’s usefulness to its next reader matters as much as its completeness as a database export.

14:5515:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:55 · section reference included

From generated documentation to informed decisions

An audience question asks how to get started generating API documentation. Sikelianos’s answer begins before the choice of documentation site: write an OpenAPI description in YAML or JSON that captures the API’s behavior. He names Docusaurus, Read the Docs, and ReadMe as examples in the documentation ecosystem. Schema-driven tooling can also produce SDKs and clients in different languages, though those capabilities are not necessarily native to every named product. The reusable asset is the API description.

The final question extends the discussion from integration to discovery and distribution. If assistants increasingly inform purchasing decisions—and might eventually make some of them—what does a service need to expose? Sikelianos starts with strong search and API access to the information a user needs to make an informed choice.

Pricing is his concrete example. At the time of the talk, Replicate users had to visit pricing pages or individual model pages to find it. Exposing that information as public, structured JSON would let an assistant retrieve costs and compare options inside the conversation. This is a proposed capability in the discussion, not a demonstrated pricing endpoint.

The imagined request is to compare video models such as Veo, Kling, and MiniMax by expense, speed, and output quality. Pricing data would support the cost comparison; speed and quality would require corresponding evidence of their own. No model ranking is demonstrated. The substantive requirement is that the facts needed for a decision be available in a form the assistant can retrieve and reason over. Search can find the options, but structured decision data makes those options possible to evaluate.

17:0317:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:03 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…