AI Engineer Summit 2023
Pragmatic AI With TypeChat
Loading slides…
Read the talk
Pragmatic AI With TypeChat
TypeChat connects natural-language requests to application code by using types to guide generation, validate results, and describe programs whose operations remain under application control.
From a talk by Daniel Rosenwasser
Before you start: Basic familiarity with TypeScript interfaces, JSON, and function calls will help you follow the examples.
A rainy-day recommendation needs a dependable shape
What should an app do with a request for places to visit on a rainy day in Seattle? Its interface might show a map and a list of recommendations, each with a venue name and description. A conversational answer is useful to a person, but the application needs those pieces as separate fields. This is the gap Daniel Rosenwasser, program manager on TypeScript, introduces: connecting the freedom of a chat interface to the precise data expected by traditional software.
A tempting first implementation asks the model to keep its answer short and put each recommendation on a separate line. After seeing a few responses, a developer notices a pattern: every line begins with a number, followed by a venue name, a colon, and a description. The parser practically writes itself: split on newlines, remove the leading numbers, then split on the colon.
But an observed formatting habit is not an interface contract. The model can change its formatting, and the content itself can contain a delimiter that breaks the parser. More instructions may make the pattern more common; they do not make parsing arbitrary prose dependable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
JSON supplies syntax; types supply the contract
Asking for JSON improves the situation immediately. Give the model an example object containing venue names and descriptions, and it can return something the application can parse without interpreting prose. Yet a parseable response can still contain unexpected properties or omit information the app needs. Valid JSON is only the beginning of validation.
An example also becomes an awkward way to describe a richer contract. Uniform objects are easy to illustrate; optional properties, required properties that sometimes contain null, and values that can be either strings or numbers introduce combinations that examples must somehow cover. Types express these distinctions directly:
| Requirement | TypeScript form |
|---|---|
| Property may be absent | description?: string |
| Property must exist but may be null | description: string | null |
| Value accepts two alternatives | value: string | number |
These declarations describe the allowed cases without enumerating every possible object.
For the Seattle app, the contract can remain small: a list of objects, each with two string fields. An interface expressing that shape looks like this:
typescript
interface Venue {
name: string;
description: string;
}
interface Recommendations {
venues: Venue[];
}
The model receives the user's intent together with the type definitions the application expects to consume. Those types become guidance for producing the answer. Rosenwasser's qualification is that the model needs sufficient training on both natural language and code; naming a provider or a local model does not establish that every model will perform equally well.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use the same types to validate and repair
Guiding generation solves only half the problem. The returned data must also satisfy the contract. TypeScript provides a useful route because JSON values can be represented inside a TypeScript program. Under the hood, the library constructs a miniature program that lets the compiler check the candidate against the requested type.
That produces a straightforward validation and repair procedure:
- Check the generated data against the expected type using the TypeScript compiler.
- If the check succeeds, return the well-typed data.
- If it fails, use the compiler diagnostic to ask the model for a repaired answer.
The same contract therefore guides generation and supplies concrete feedback when generation goes wrong.
TypeChat packages this process into a TypeScript library available through npm. Its usefulness grows with schemas richer than the recommendation list: a coffee order, for example, can describe several kinds of items and the choices available for each. User intent and those types go into translation; structured, checked output comes back.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give unsupported requests somewhere to go
The live demonstration starts in a cloned TypeChat repository with dependencies installed. Its README orders the examples by increasing complexity, beginning with positive, negative, or neutral sentiment classification. The coffee shop moves beyond that introductory task: its schema describes orders, and its command-line prompt accepts a request for one latte with foam. That request translates successfully.
The next request combines a latte with a medium purple gorilla named Bonsai. Translation still succeeds, but success does not mean the coffee shop has acquired a gorilla. The JSON carries the unsupported phrase through an unknown-text alternative, allowing the application to tell the user which part it did not understand. The schema can represent both the recognized order and the unresolved part of the request.
This shifts some of the work from prompt engineering to schema engineering. The schema does more than list acceptable products: it defines how the application can receive and handle an incomplete interpretation. An explicit unknown case gives the model a way to preserve unsupported intent instead of forcing every phrase into a menu item.
Rosenwasser describes the shown coffee application code as under 40 lines. Its core wiring creates a model from environment settings, constructs a JSON translator with the type-definition contents and expected type name, then calls translate for each request. The recording uses the original createJsonTranslator(model, schema, typeName) construction; TypeChat 0.1.0 later changed this to createJsonTranslator(model, validator) and moved program helpers to typechat/ts. The line-count claim concerns the shown application wiring, not its separate schema or the library underneath it.
Removing the unknown-input alternative has an immediate effect in the editor: the application code that handles that case now produces type errors. After adjusting the example, Rosenwasser requests a cappuccino and a purple gorilla named Bonsai. This time, the result includes a bagel with butter. The altered schema has left the model without an appropriate way to express that part of the request.
Type correctness does not establish that an answer faithfully represents the user's intent. A bagel can be a valid menu item while being the wrong interpretation. A schema that anticipates misunderstanding gives the application a recovery path: show what was recognized, expose what remains unclear, and let the user resolve it. The unknown case supports that interaction; it is not a guarantee against every mistaken interpretation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A list of commands eventually needs dataflow
Structured data can also describe actions. A request to schedule an appointment can translate into a calendar command, and several requested actions can become a list of commands. This is a natural extension of the coffee example: the output now tells the application what to do.
A simple chain works when one operation's output becomes the next operation's input. The scripting slide illustrates a sequence of readFile, trimText, and writeFile. But what happens when an operation needs several arguments, or two later operations need the same earlier result? The representation needs a way to name and reuse values. A command list is becoming a program.
One approach is to declare the methods the model may call and ask it to generate code using only that API. Execution constraints quickly complicate the idea. To protect availability, the application might forbid loops; it might also exclude lambdas and other constructs. A model trained on full JavaScript or Python can still generate code outside that permitted subset, leaving the application with a rejected program instead of a usable result. Sandboxing and the distinction between synchronous and asynchronous APIs add further concerns.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Represent the program as JSON
TypeChat's experimental approach is to generate a small artificial language encoded as JSON. The API types still guide generation, but the model produces a structured program rather than unrestricted source code. References identify results from prior steps, making those results available as arguments to later operations. Rosenwasser compares this representation to an abstract syntax tree and to static single assignment form: both analogies help explain explicit operations and dependencies.
The JSON program is then used to construct a synthetic TypeScript program in memory. Compiler validation checks both the calls and the connections between them: an operation must be available in the declared API, and a referenced result must have a type accepted by the consuming operation. This goes beyond checking that a command name exists. It checks whether the proposed program fits the API's type relationships. Those checks constrain the program; they do not by themselves establish a complete execution sandbox.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let application code perform the arithmetic
The calculator demonstration makes the dependency concrete: add one to forty-one, then divide by seven. The addition produces the value needed by the division. The model's job is to describe those operations and their relationship; application code performs the arithmetic. The resulting calculation is (1 + 41) / 7 = 6, not evidence that the model itself has become a more accurate calculator.
Rosenwasser describes the shown calculator application code as under 50 lines. It uses a program translator, stops if translation and validation fail, and otherwise passes the validated program to an evaluator. The math application's evaluator callback acts as an instruction interpreter, dispatching each operation to application code. That dispatch can use an object, a function, or a switch.
For the two operations in this request, an application-owned dispatcher can be as small as:
typescript
type MathOperation = "add" | "divide";
function dispatch(
operation: MathOperation,
left: number,
right: number,
): number {
switch (operation) {
case "add":
return left + right;
case "divide":
return left / right;
}
}
const sum = dispatch("add", 1, 41);
const result = dispatch("divide", sum, 7);
In the translated workflow, the evaluator resolves the reference to the addition's result before invoking the division callback. This separates interpreting the request from executing its operations, while allowing tasks richer than a single command.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Carry the typed approach into Python
Other approaches overlap with parts of this design. The capability Rosenwasser emphasizes is validation of both structured JSON results and programs. The next question is whether the same idea can work beyond TypeScript. In the Python experiments, a coffee-order schema again guides interpretation, while a calculator API describes operations as class methods with typed arguments and explanatory comments. Rosenwasser reports encouraging results, without presenting a quantitative benchmark.
The Python code view shows calculator methods and their docstrings, illustrating how an API can communicate both permissible operations and their intended meaning. The next example extends that idea from arithmetic to a richer CSV-processing API.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Compose a data-cleaning request from available operations
The CSV API exposes column access and operations that support filtering and joins, with Boolean selections identifying rows. The closing request is to read a CSV, find values equal to N/A, and drop the corresponding rows. The same program-generation idea now connects several data-processing operations: obtain the data, compute a selection, and use that selection to remove rows.
The Python work shown here is explicitly a prototype, not ready for prime time. Even so, it extends the central mechanism from menu items to arithmetic and tabular data: describe the application's vocabulary with types, translate intent into that vocabulary, and validate the result before the application uses it.
Rosenwasser closes by asking engineers to try TypeChat and share what they are building. The practical goal is to bring language models into the precise world of everyday applications, giving developers a familiar contract through which free-form requests can become usable data and executable plans.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Library and examples for translating natural-language requests into validated structured data.
Application code that translates requests into programs and dispatches arithmetic operations through an evaluator callback.
Further reading
- Introducing TypeChatArticle
The original announcement explains type-guided generation, compiler validation and repair.
Setup instructions and an ordered path through sentiment, coffee, calendar and program-generation examples.
- Schema engineering techniquesDocumentation
Practical schema guidance, including explicit ways to represent requests outside an application's domain.
- PypechatRepository
Daniel Rosenwasser's Python prototype with coffee, math and CSV examples.
Updates since the talk
- TypeChat 0.1.0 API changesArticle
Explains validator-based translator construction, Zod support and changed import locations introduced after the talk.
Full transcript
Loading transcript…