AI Engineer World's Fair 2024

Pydantic is STILL all you need

Jason Liu· Independent15:21

Loading slides…

Read the talk

Pydantic is still all you need

Typed responses turn model output into ordinary application data: objects you can validate, stream, search with, render and send to an execution engine.

From a talk by Jason Liu

Before you start: Familiarity with Python type hints, basic Pydantic models and chat-model API calls will help you follow the examples.

An API should return more than a string

Imagine hiring an intern to write an API and receiving a string that you must feed into json.loads, followed by a hopeful check that the expected fields exist. Jason Liu’s response is to replace the intern with Devin and ask it to use FastAPI and Pydantic. The joke opens a practical question: why accept this interface from a language model when it would be unacceptable from an ordinary API?

Gold text on black describes an intern's API returning a string that requires json.loads and hoping the expected data is present.Open full source frame
An API that returns a string leaves callers parsing and hoping the data is correct.

Parsing and repairing strings was familiar work with GPT-3, but application developers already have better tools for representing data. Pydantic in Python, Ecto in Elixir and Active Record offer established ways to work with structured records. Without schemas, an integration loses compatibility, composability and reliability. Liu’s earlier Pydantic talk proposed function calling and typed responses as the alternative: nested models express modular structures, while validators define additional conditions the returned data must satisfy.

0:250:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:25 · section reference included

Instructor grows around a small interface

The installation remains pip install instructor. By this talk, Instructor had reached 1.0, launched implementations in Python, TypeScript, Ruby, Go and Elixir, and added a newly built Rust version. Liu describes its value as roughly 600 lines of code that application developers do not want to write themselves. Liu reports 40% month-over-month growth for the Python library and downloads around 2% of OpenAI’s library downloads. He does not specify the measurement window or counting methodology.

The major-version change was modest: the client setup moved from instructor.patch to instructor.from_openai. Provider coverage expanded to Ollama, LlamaCPP, Anthropic, Cohere, Gemini and Groq. The intended stability comes from keeping the application contract small: as models gain function-calling capabilities, the same typed interface can sit above them.

1:542:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:54 · section reference included

Return an object, stream objects, or stream a partial

Start by defining a Pydantic model, wrapping the model client and passing response_model=User. This resembles FastAPI’s response-model interface: the model declaration determines what the caller receives. Improved typing also lets the IDE infer a User return value and flag incorrect field access. A Python call using that interface looks like this, with the provider’s model identifier supplied through configuration:

python

import os

import instructor
from openai import OpenAI
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(OpenAI())

user = client.chat.completions.create(
    model=os.environ["OPENAI_MODEL"],
    response_model=User,
    messages=[
        {"role": "user", "content": "Jason is 30 years old."}
    ],
)

print(user.name, user.age)

The application works with fields on an object instead of extracting a JSON string from the provider response.

When the input contains two users, iterable streaming returns each completed User as it becomes available. With stream=True, the application can process the first object before the full extraction finishes, reducing the wait for useful output. The inspected slide uses create_iterable and shows a generator of typed users.

Slide titled Iterables shows a User generator type tooltip, a create_iterable call with stream=True, and a loop printing each user.Open full source frame
The iterable example streams typed User objects.

Partial streaming addresses a different problem: rendering one evolving object, such as the structure behind a generative UI. It avoids making the application maintain its own JSON parsing stack while tokens arrive. Liu describes this in terms of whole-object validation; current partial-response documentation instead describes incremental snapshots with optionalized fields and explicitly says validators are unsupported during streaming. An intermediate snapshot should therefore not be treated as a final object that has passed every validator.

Liu summarizes the interface as one noun—the client—and three verbs, spoken as create, create_with_iterable and create_with_partial. The naming differs from the slide’s create_iterable and today’s create_partial, but the three consumption patterns are straightforward:

PatternWhat the caller consumes
CreateOne completed model
IterableCompleted models as they arrive
PartialSuccessive snapshots of one model

The remaining application decisions live in the response model, validators and messages array. Features that fit within that messages interface need not introduce another application abstraction.

2:583:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:58 · section reference included

Validation errors become conditional instructions

Generation, RAG and extraction all benefit from the same shift: program with data structures and explicit rules. The principle also applies to vision and agents, although Liu warns that models differ in how well they respond to retry feedback. The programming language has not changed; the task is to bring familiar programming techniques back into the model interaction.

A field validator can define correctness without putting every rule into the initial prompt. In Liu’s example, the prompt does not request uppercase names. Python checks the returned name and raises an informative error if it is not uppercase:

python

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str

    @field_validator("name")
    @classmethod
    def require_uppercase(cls, value: str) -> str:
        if value != value.upper():
            raise ValueError("Name must be uppercase.")
        return value

With retrying enabled, Instructor catches the validation failure and supplies the error message to the model for another attempt. The validator rejects rather than silently uppercasing the value: its message becomes a conditionally added instruction. Liu’s displayed result is JASON; he describes it as appearing after one API call, without spelling out the underlying retry accounting.

4:394:49
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:39 · section reference included

Validate relationships across receipt fields

A receipt requires more than independently valid fields. In a Ramp-style processing flow, a vision model extracts product prices, quantities and a total cost. A model-level validator checks whether those fields agree. For a receipt model whose total is defined as the sum of its line items, the rule can be expressed directly:

python

from decimal import Decimal
from typing import Self

from pydantic import BaseModel, Field, model_validator

class Product(BaseModel):
    price: Decimal = Field(ge=0)
    quantity: int = Field(gt=0)

class Receipt(BaseModel):
    products: list[Product]
    total_cost: Decimal

    @model_validator(mode="after")
    def check_total(self) -> Self:
        expected = sum(
            (item.price * item.quantity for item in self.products),
            Decimal("0"),
        )
        if expected != self.total_cost:
            raise ValueError(
                f"Line items sum to {expected}; "
                f"total_cost is {self.total_cost}."
            )
        return self

This validates a relationship among extracted values, not just their individual types.

Liu says this receipt discrepancy does not occur in 99% of cases, without supplying an evaluation. His concern is what happens when it does: an explicit validation failure can appear as a red bar in Datadog instead of quietly becoming accepted data. Re-asking then gives the model an opportunity to correct the inconsistency.

6:066:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:06 · section reference included

Give generated answers a useful shape

An ordinary chat response is already an object, but its useful payload is often just a content string. That leaves the application responsible for parsing anything more elaborate. A structured RAG answer can instead contain both answer content and a list of follow-up questions. Those questions can be grounded in the supplied context, exposing other things the system is equipped to answer.

Liu also describes a production validator that extracts URLs with a regular expression and checks whether they return HTTP 200. He describes the request method as POST. When a check fails, the next attempt is instructed to omit the link rather than keep trying to invent a replacement. This catches some bad links, but HTTP success means the particular request succeeded; it does not establish that the destination supports the generated claim. POST also has different semantics from retrieving a page.

6:416:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:41 · section reference included

Turn search intent into a typed request

Consider a request for the latest news from Z. Semantic similarity alone does not enforce recency. BM25 may help match the source, and different sources may require different indices, but the application still needs to represent the user’s constraints explicitly. Liu introduces a Search object that makes those decisions inspectable:

FieldPurpose
querySearch text
start_dateBeginning of the date range
end_dateOptional end of the range
limitRequested result count, such as five
sourceBackend or index to query

The language model translates intent into these fields; the search implementation knows how to apply them.

Applications in RAG slide shows a Search model with query, start and end dates, limit, and source fields, followed by a function returning that model.Open full source frame
A Search model makes query constraints explicit.

A create_search function takes a string and returns the search object. Both the model API call and the mechanics of querying the chosen endpoint can remain behind that boundary. Structured requests also make behavior measurable: the application can validate date ranges and inspect how often the model produces zero-day, one-day or longer intervals.

7:487:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:48 · section reference included

Compose retrieval with two models and two functions

A comparison question—what is the difference between X and Y?—can require more than one search. Iterable output lets the model produce a search request for each subject. The application can execute those requests in parallel and collect their results. Liu organizes the model-facing portion around two data models, search requests and answers, and two functions that return them.

The resulting RAG flow is ordinary orchestration:

  1. Pass the question to create_search and obtain multiple search objects.
  2. Execute each search against its selected backend.
  3. Gather the retrieved context.
  4. Pass the question and context to answer_question.
  5. Render the structured answer.

An OpenAPI endpoint or React interface can consume the resulting structure without taking over the model interaction. The LLM sits behind a typed boundary. That boundary guarantees conformance to declared types and validation constraints, not the factual truth of every answer.

8:549:09
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:54 · section reference included

Extract labels and meeting records

Classification can begin with a single constrained field: label: Literal["spam", "not spam"]. The schema restricts the accepted result to the application’s label set instead of accepting explanatory prose where a label belongs. Liu reports about a 15% classifier accuracy improvement from adding chain of thought. He supplies no dataset, model, baseline or evaluation conditions, and does not distinguish relative percent from percentage points. The concrete schema lesson is that a reasoning field can coexist with a tightly constrained classification field.

Meeting extraction extends the same approach to a richer record: meeting type, title, action items and summary. An action item’s owner may initially be a string, but a domain validator could require that owner to be one of the participants. This moves correctness beyond the shape of a record toward whether its values belong to the application’s known entities.

9:5710:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:57 · section reference included

An image table becomes a DataFrame

A Google Calendar integration could supply the participant list for that ownership check. The next extraction example pushes the type boundary further: a Table has a caption string and a field whose application-facing value is a pandas DataFrame, even though the model produces a Markdown table from an image.

The mechanism combines a native Python type with parsing, serialization and schema metadata. Annotated, standardized in PEP 593, provides the metadata mechanism; Liu discusses annotations without naming the proposal number. The Markdown DataFrame pattern gives each layer a distinct job:

  • Native type: InstanceOf[pd.DataFrame] describes the runtime object and enables DataFrame method completion.
  • Input conversion: BeforeValidator parses the model’s Markdown into a DataFrame before the instance check.
  • Output conversion: PlainSerializer turns that DataFrame back into Markdown when serialized.
  • Model-facing schema: WithJsonSchema describes the string representation the model should generate.

The model and the application can therefore use different representations of the same table.

For simple pipe-delimited tables without escaped pipes or multiline cells, the pattern can be written as follows:

python

from typing import Annotated

import pandas as pd
from pydantic import (
    BaseModel,
    BeforeValidator,
    InstanceOf,
    PlainSerializer,
    WithJsonSchema,
)

def parse_markdown(value: str | pd.DataFrame) -> pd.DataFrame:
    if isinstance(value, pd.DataFrame):
        return value
    rows = [
        [cell.strip() for cell in line.strip().strip("|").split("|")]
        for line in value.strip().splitlines()
        if line.strip()
    ]
    if len(rows) < 2:
        raise ValueError("Expected a Markdown header and separator row.")
    header, separator, *data = rows
    if len(separator) != len(header) or any(
        not cell or set(cell) - set("-:") or "-" not in cell
        for cell in separator
    ):
        raise ValueError("Invalid Markdown separator row.")
    if any(len(row) != len(header) for row in data):
        raise ValueError("Every row must match the header width.")
    return pd.DataFrame(data, columns=header)

MarkdownDataFrame = Annotated[
    InstanceOf[pd.DataFrame],
    BeforeValidator(parse_markdown),
    PlainSerializer(
        lambda frame: frame.to_markdown(index=False),
        return_type=str,
    ),
    WithJsonSchema({
        "type": "string",
        "description": "A Markdown table with a header and separator row.",
    }),
]

class Table(BaseModel):
    caption: str
    data: MarkdownDataFrame

table = Table.model_validate({
    "caption": "Products",
    "data": "| Product | Quantity |\n| --- | --- |\n| Pen | 2 |",
})
csv_text = table.data.to_csv(index=False)

Here the small product table illustrates the conversion. Once validation returns a Table, its data field already exposes to_csv; downstream code does not need to repeat the model-output parsing. Markdown serialization through pandas requires its tabulate dependency.

10:4610:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:46 · section reference included

Reliability depends on useful feedback

The same approach can generate date ranges, relationships, knowledge graphs, DAGs, workflows and tables. The design work lies in choosing a useful response model and writing validators for it. As models improve, Liu expects those interfaces to require less intervention.

Liu reports that one retry is often enough with OpenAI and Anthropic models when validation errors are informative. He presents this as practical experience rather than a measured success rate. A good error message now serves two readers: the developer diagnosing the failure and the model attempting a correction. Faster models also make the extra round trip easier to afford; Liu refers to newer “3.5” and “4.0” models while discussing that latency tradeoff.

In consulting work involving complex validations, Liu reports that function-calling fine-tuning reduced failure rates from 4–5% to zero on models such as Mistral and GPT-3.5. The slide describes the result as near-zero; no evaluation size, model versions or held-out conditions are supplied. The observation supports testing fine-tuning when failures persist, but does not establish a general zero-failure guarantee.

Learnings slide lists complex-validation observations and type-generation failures, including a 4–5% production failure rate and reduction to near-zero with fine-tuning.Open full source frame
Reliability lessons cover retries, error messages, latency, and fine-tuning.
12:0312:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:03 · section reference included

Generate reports and plans for execution

Vision, RAG and agents all need types that the application can program against. Prompt construction, the machinery implementing a response model, and constrained sampling in systems such as LlamaCPP, Ollama or Outlines sit beneath that application contract. Their implementation can change while the programmer continues to consume useful objects.

That separation opens up two applications beyond a chat answer:

  • Enterprise RAG: Generate reports that support decisions, rather than treating retrieval only as a question-answering feature.
  • Agents: Generate workflows and directed acyclic graphs, then pass those plans to an execution engine that performs the computation.

A generated DAG is a plan, not an executed outcome. The application owns its execution instead of depending on an open-ended ReAct loop to terminate.

13:1913:31
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:19 · section reference included

Keep ownership of the program

The destination is familiar software engineering: types the IDE understands, field access it can check, and errors it can underline. Generative AI becomes a producer of data structures. Developers own the objects they define, the functions they implement and the control flow that connects them. They also retain the prompt through the messages array.

Typed boundaries make generated data usable by existing software. Liu describes this as making software 3.0 backward compatible with what developers already build. The model can remain powerful and flexible without making the surrounding program mysterious: its output enters a world of objects, validation and explicit control flow.

14:1214:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:12 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript

Loading transcript…