AI Engineer World's Fair 2024
Pydantic is STILL all you need
Loading slides…
Read the talk
Pydantic is still all you need
Typed responses turn model output into ordinary application data: objects you can validate, stream, search with, render and send to an execution engine.
From a talk by Jason Liu
Before you start: Familiarity with Python type hints, basic Pydantic models and chat-model API calls will help you follow the examples.
An API should return more than a string
Imagine hiring an intern to write an API and receiving a string that you must feed into json.loads, followed by a hopeful check that the expected fields exist. Jason Liu’s response is to replace the intern with Devin and ask it to use FastAPI and Pydantic. The joke opens a practical question: why accept this interface from a language model when it would be unacceptable from an ordinary API?
Parsing and repairing strings was familiar work with GPT-3, but application developers already have better tools for representing data. Pydantic in Python, Ecto in Elixir and Active Record offer established ways to work with structured records. Without schemas, an integration loses compatibility, composability and reliability. Liu’s earlier Pydantic talk proposed function calling and typed responses as the alternative: nested models express modular structures, while validators define additional conditions the returned data must satisfy.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Instructor grows around a small interface
The installation remains pip install instructor. By this talk, Instructor had reached 1.0, launched implementations in Python, TypeScript, Ruby, Go and Elixir, and added a newly built Rust version. Liu describes its value as roughly 600 lines of code that application developers do not want to write themselves. Liu reports 40% month-over-month growth for the Python library and downloads around 2% of OpenAI’s library downloads. He does not specify the measurement window or counting methodology.
The major-version change was modest: the client setup moved from instructor.patch to instructor.from_openai. Provider coverage expanded to Ollama, LlamaCPP, Anthropic, Cohere, Gemini and Groq. The intended stability comes from keeping the application contract small: as models gain function-calling capabilities, the same typed interface can sit above them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Return an object, stream objects, or stream a partial
Start by defining a Pydantic model, wrapping the model client and passing response_model=User. This resembles FastAPI’s response-model interface: the model declaration determines what the caller receives. Improved typing also lets the IDE infer a User return value and flag incorrect field access. A Python call using that interface looks like this, with the provider’s model identifier supplied through configuration:
python
import os
import instructor
from openai import OpenAI
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_openai(OpenAI())
user = client.chat.completions.create(
model=os.environ["OPENAI_MODEL"],
response_model=User,
messages=[
{"role": "user", "content": "Jason is 30 years old."}
],
)
print(user.name, user.age)
The application works with fields on an object instead of extracting a JSON string from the provider response.
When the input contains two users, iterable streaming returns each completed User as it becomes available. With stream=True, the application can process the first object before the full extraction finishes, reducing the wait for useful output. The inspected slide uses create_iterable and shows a generator of typed users.
Partial streaming addresses a different problem: rendering one evolving object, such as the structure behind a generative UI. It avoids making the application maintain its own JSON parsing stack while tokens arrive. Liu describes this in terms of whole-object validation; current partial-response documentation instead describes incremental snapshots with optionalized fields and explicitly says validators are unsupported during streaming. An intermediate snapshot should therefore not be treated as a final object that has passed every validator.
Liu summarizes the interface as one noun—the client—and three verbs, spoken as create, create_with_iterable and create_with_partial. The naming differs from the slide’s create_iterable and today’s create_partial, but the three consumption patterns are straightforward:
| Pattern | What the caller consumes |
|---|---|
| Create | One completed model |
| Iterable | Completed models as they arrive |
| Partial | Successive snapshots of one model |
The remaining application decisions live in the response model, validators and messages array. Features that fit within that messages interface need not introduce another application abstraction.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Validation errors become conditional instructions
Generation, RAG and extraction all benefit from the same shift: program with data structures and explicit rules. The principle also applies to vision and agents, although Liu warns that models differ in how well they respond to retry feedback. The programming language has not changed; the task is to bring familiar programming techniques back into the model interaction.
A field validator can define correctness without putting every rule into the initial prompt. In Liu’s example, the prompt does not request uppercase names. Python checks the returned name and raises an informative error if it is not uppercase:
python
from pydantic import BaseModel, field_validator
class User(BaseModel):
name: str
@field_validator("name")
@classmethod
def require_uppercase(cls, value: str) -> str:
if value != value.upper():
raise ValueError("Name must be uppercase.")
return value
With retrying enabled, Instructor catches the validation failure and supplies the error message to the model for another attempt. The validator rejects rather than silently uppercasing the value: its message becomes a conditionally added instruction. Liu’s displayed result is JASON; he describes it as appearing after one API call, without spelling out the underlying retry accounting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Validate relationships across receipt fields
A receipt requires more than independently valid fields. In a Ramp-style processing flow, a vision model extracts product prices, quantities and a total cost. A model-level validator checks whether those fields agree. For a receipt model whose total is defined as the sum of its line items, the rule can be expressed directly:
python
from decimal import Decimal
from typing import Self
from pydantic import BaseModel, Field, model_validator
class Product(BaseModel):
price: Decimal = Field(ge=0)
quantity: int = Field(gt=0)
class Receipt(BaseModel):
products: list[Product]
total_cost: Decimal
@model_validator(mode="after")
def check_total(self) -> Self:
expected = sum(
(item.price * item.quantity for item in self.products),
Decimal("0"),
)
if expected != self.total_cost:
raise ValueError(
f"Line items sum to {expected}; "
f"total_cost is {self.total_cost}."
)
return self
This validates a relationship among extracted values, not just their individual types.
Liu says this receipt discrepancy does not occur in 99% of cases, without supplying an evaluation. His concern is what happens when it does: an explicit validation failure can appear as a red bar in Datadog instead of quietly becoming accepted data. Re-asking then gives the model an opportunity to correct the inconsistency.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give generated answers a useful shape
An ordinary chat response is already an object, but its useful payload is often just a content string. That leaves the application responsible for parsing anything more elaborate. A structured RAG answer can instead contain both answer content and a list of follow-up questions. Those questions can be grounded in the supplied context, exposing other things the system is equipped to answer.
Liu also describes a production validator that extracts URLs with a regular expression and checks whether they return HTTP 200. He describes the request method as POST. When a check fails, the next attempt is instructed to omit the link rather than keep trying to invent a replacement. This catches some bad links, but HTTP success means the particular request succeeded; it does not establish that the destination supports the generated claim. POST also has different semantics from retrieving a page.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn search intent into a typed request
Consider a request for the latest news from Z. Semantic similarity alone does not enforce recency. BM25 may help match the source, and different sources may require different indices, but the application still needs to represent the user’s constraints explicitly. Liu introduces a Search object that makes those decisions inspectable:
| Field | Purpose |
|---|---|
query | Search text |
start_date | Beginning of the date range |
end_date | Optional end of the range |
limit | Requested result count, such as five |
source | Backend or index to query |
The language model translates intent into these fields; the search implementation knows how to apply them.
A create_search function takes a string and returns the search object. Both the model API call and the mechanics of querying the chosen endpoint can remain behind that boundary. Structured requests also make behavior measurable: the application can validate date ranges and inspect how often the model produces zero-day, one-day or longer intervals.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Compose retrieval with two models and two functions
A comparison question—what is the difference between X and Y?—can require more than one search. Iterable output lets the model produce a search request for each subject. The application can execute those requests in parallel and collect their results. Liu organizes the model-facing portion around two data models, search requests and answers, and two functions that return them.
The resulting RAG flow is ordinary orchestration:
- Pass the question to
create_searchand obtain multiple search objects. - Execute each search against its selected backend.
- Gather the retrieved context.
- Pass the question and context to
answer_question. - Render the structured answer.
An OpenAPI endpoint or React interface can consume the resulting structure without taking over the model interaction. The LLM sits behind a typed boundary. That boundary guarantees conformance to declared types and validation constraints, not the factual truth of every answer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Extract labels and meeting records
Classification can begin with a single constrained field: label: Literal["spam", "not spam"]. The schema restricts the accepted result to the application’s label set instead of accepting explanatory prose where a label belongs. Liu reports about a 15% classifier accuracy improvement from adding chain of thought. He supplies no dataset, model, baseline or evaluation conditions, and does not distinguish relative percent from percentage points. The concrete schema lesson is that a reasoning field can coexist with a tightly constrained classification field.
Meeting extraction extends the same approach to a richer record: meeting type, title, action items and summary. An action item’s owner may initially be a string, but a domain validator could require that owner to be one of the participants. This moves correctness beyond the shape of a record toward whether its values belong to the application’s known entities.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An image table becomes a DataFrame
A Google Calendar integration could supply the participant list for that ownership check. The next extraction example pushes the type boundary further: a Table has a caption string and a field whose application-facing value is a pandas DataFrame, even though the model produces a Markdown table from an image.
The mechanism combines a native Python type with parsing, serialization and schema metadata. Annotated, standardized in PEP 593, provides the metadata mechanism; Liu discusses annotations without naming the proposal number. The Markdown DataFrame pattern gives each layer a distinct job:
- Native type:
InstanceOf[pd.DataFrame]describes the runtime object and enables DataFrame method completion. - Input conversion:
BeforeValidatorparses the model’s Markdown into a DataFrame before the instance check. - Output conversion:
PlainSerializerturns that DataFrame back into Markdown when serialized. - Model-facing schema:
WithJsonSchemadescribes the string representation the model should generate.
The model and the application can therefore use different representations of the same table.
For simple pipe-delimited tables without escaped pipes or multiline cells, the pattern can be written as follows:
python
from typing import Annotated
import pandas as pd
from pydantic import (
BaseModel,
BeforeValidator,
InstanceOf,
PlainSerializer,
WithJsonSchema,
)
def parse_markdown(value: str | pd.DataFrame) -> pd.DataFrame:
if isinstance(value, pd.DataFrame):
return value
rows = [
[cell.strip() for cell in line.strip().strip("|").split("|")]
for line in value.strip().splitlines()
if line.strip()
]
if len(rows) < 2:
raise ValueError("Expected a Markdown header and separator row.")
header, separator, *data = rows
if len(separator) != len(header) or any(
not cell or set(cell) - set("-:") or "-" not in cell
for cell in separator
):
raise ValueError("Invalid Markdown separator row.")
if any(len(row) != len(header) for row in data):
raise ValueError("Every row must match the header width.")
return pd.DataFrame(data, columns=header)
MarkdownDataFrame = Annotated[
InstanceOf[pd.DataFrame],
BeforeValidator(parse_markdown),
PlainSerializer(
lambda frame: frame.to_markdown(index=False),
return_type=str,
),
WithJsonSchema({
"type": "string",
"description": "A Markdown table with a header and separator row.",
}),
]
class Table(BaseModel):
caption: str
data: MarkdownDataFrame
table = Table.model_validate({
"caption": "Products",
"data": "| Product | Quantity |\n| --- | --- |\n| Pen | 2 |",
})
csv_text = table.data.to_csv(index=False)
Here the small product table illustrates the conversion. Once validation returns a Table, its data field already exposes to_csv; downstream code does not need to repeat the model-output parsing. Markdown serialization through pandas requires its tabulate dependency.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reliability depends on useful feedback
The same approach can generate date ranges, relationships, knowledge graphs, DAGs, workflows and tables. The design work lies in choosing a useful response model and writing validators for it. As models improve, Liu expects those interfaces to require less intervention.
Liu reports that one retry is often enough with OpenAI and Anthropic models when validation errors are informative. He presents this as practical experience rather than a measured success rate. A good error message now serves two readers: the developer diagnosing the failure and the model attempting a correction. Faster models also make the extra round trip easier to afford; Liu refers to newer “3.5” and “4.0” models while discussing that latency tradeoff.
In consulting work involving complex validations, Liu reports that function-calling fine-tuning reduced failure rates from 4–5% to zero on models such as Mistral and GPT-3.5. The slide describes the result as near-zero; no evaluation size, model versions or held-out conditions are supplied. The observation supports testing fine-tuning when failures persist, but does not establish a general zero-failure guarantee.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate reports and plans for execution
Vision, RAG and agents all need types that the application can program against. Prompt construction, the machinery implementing a response model, and constrained sampling in systems such as LlamaCPP, Ollama or Outlines sit beneath that application contract. Their implementation can change while the programmer continues to consume useful objects.
That separation opens up two applications beyond a chat answer:
- Enterprise RAG: Generate reports that support decisions, rather than treating retrieval only as a question-answering feature.
- Agents: Generate workflows and directed acyclic graphs, then pass those plans to an execution engine that performs the computation.
A generated DAG is a plan, not an executed outcome. The application owns its execution instead of depending on an open-ended ReAct loop to terminate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep ownership of the program
The destination is familiar software engineering: types the IDE understands, field access it can check, and errors it can underline. Generative AI becomes a producer of data structures. Developers own the objects they define, the functions they implement and the control flow that connects them. They also retain the prompt through the messages array.
Typed boundaries make generated data usable by existing software. Liu describes this as making software 3.0 backward compatible with what developers already build. The model can remain powerful and flexible without making the surrounding program mysterious: its output enters a world of objects, validation and explicit control flow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A structured-generation library for constraining language-model output formats.
Further reading
- Instructor 1.0 announcementArticle
Jason Liu explains the 1.0 client interface, typing improvements and validation support. Embedded examples have since been updated.
Build a DataFrame type that parses Markdown, serializes back to Markdown and supplies a model-facing JSON schema.
- PEP 593: Annotated typesDocumentation
The Python proposal defining metadata attached to type annotations through Annotated.
Define typed models and understand what Pydantic validation guarantees.
Updates since the talk
Current examples of incremental structured output, including limitations on validators during streaming.
Configure retries and handle errors in current Instructor applications.
Full transcript
Loading transcript…