← All speakers

Jason Liu

Conference affiliation: Developer Experience, OpenAI · OpenAI · 2026

Jason Liu is the creator of Instructor, the open-source library that turns language-model responses into typed, validated objects, and a developer experience engineer on OpenAI’s Codex team. Instructor lets developers specify the data they need with familiar Python models, check business requirements in code, and return informative errors to the model when a response fails validation.

On this page

Building recommendation systems

Jason is based in San Francisco. He attended the University of Waterloo, where he co-founded and led the Waterloo Data Science Club. His early career included research at New York University, data-science work at ActionIQ, and a 2017 internship at Facebook, now Meta, developing algorithms to identify harmful content and coordinated malicious activity. After an earlier placement at Stitch Fix, he returned in February 2018 and spent five years working on recommendations and computer vision.

At Stitch Fix, Jason architected Flight, a recommendation framework that separated recommendation pipelines, reusable operators, and execution infrastructure. Teams could assemble recommendation logic from operators while the infrastructure handled execution. Operator-level errors, logging, and tracing helped engineers locate failures within a workflow. His work also included vector-based product similarity using CLIP, GPT-3 embeddings, and FAISS; computer-vision research and dataset development; and language-model tools to help stylists write faster and improve writing quality.

In March 2023, Jason founded 567 Studios and developed a solo consulting and teaching practice around AI products. He worked with companies including Zapier, HubSpot, Weights & Biases, Modal, Timescale, and Pydantic. He has since closed the consulting and training offerings, leaving his Systematically Improving RAG course materials publicly available. The materials connect evaluation and user feedback to specialized retrieval, routing, and product prioritization. Alongside his engineering work, he is an angel investor and scout for Andreessen Horowitz.

Instructor: an interface developers can program with

Instructor became Jason’s defining open-source contribution. A Pydantic model describes the desired response through field types, descriptions, nested objects, and validation rules. Instructor handles parsing, validation, and retries around the language-model call, returning objects that the rest of an application can use. Field descriptions become part of the schema sent to the model, while types give developers autocomplete and type checking in their editor.

The interface itself needed revision. Instructor initially patched the OpenAI client, a convenient way to add structured extraction that made correct type inference difficult. With Instructor 1.0 in April 2024, Jason introduced a client interface that improved typing and provided explicit methods for streaming partial objects and iterables. He kept it close to existing provider APIs so developers could continue using their model settings and messages. Instructor expanded beyond its initial OpenAI integration into multiple providers and programming languages. Jason’s account of the project notes that OpenAI cited it as inspiration for its Structured Outputs feature.

Two applications explain the substance of his approach:

  • Validation with actionable errors. A receipt extractor can check whether quantities and prices add up to the reported total. If they do not, the validation error tells the model what needs correction on the next attempt. Jason’s structured-output examples put these business rules into ordinary validation code. His earlier work also checks that supporting quotations occur in the supplied text and excludes facts without a matching quotation. These checks establish specific properties of an answer; they do not guarantee that every generated claim is true.
  • Search plans that ordinary code can execute. A model can return search requests containing sources, date ranges, and dependencies between questions. Application code chooses the backend, runs independent searches in parallel, and waits for dependent results. Jason uses the model to generate the plan’s data structure while leaving execution and control flow with the developer.

Evaluation that changes what a team builds

Jason’s consulting and teaching address a different problem: deciding what to improve once an AI application is in use. His RAG materials treat retrieval, feedback collection, and product design as parts of the same improvement process. A request for recent information may need a time filter. A question about a table may need a different retrieval path from a question about prose. Better answer generation cannot supply documents that retrieval missed.

His open-source Kura project applies this approach to conversation data. Inspired by Anthropic’s Clio, it summarizes interactions, groups them by semantic similarity, and organizes the groups into a hierarchy. Its design aims to reveal population-level patterns while limiting exposure of individual conversations.

In his joint presentation with Jeff Huber, Jason develops impact-weighted product evaluation: compare how often people request a capability with how well the application handles it. A heavily used capability with poor results deserves repair. A strong capability that users rarely discover may need better onboarding. A cluster of failures involving dates may call for a missing filter rather than a new model. Conversation summaries can capture requests, tool errors, frustration, and retry patterns that a single aggregate score obscures.

He recommends reading conversations manually while their volume is manageable, then using models to help extract and group information as the collection grows. The purpose of clustering is to form testable hypotheses about what to build, fix, or leave alone.

Jason also coauthored What We Learned from a Year of Building with LLMs with Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, and Shreya Shankar. In his identifiable operational segment, he used satirical advice about continually buying more tools and hiring specialists prematurely to criticize decisions made without understanding the business’s needs. The broader work combined the six authors’ strategic, operational, and tactical lessons.

Codex and work that continues tomorrow

Jason joined OpenAI in February 2026. His Codex work includes prototyping, education, partnerships, and experiments with delegating knowledge work. His workflows gather project context, route support issues, prepare documents, and return to unresolved tasks when another person responds.

His personal Codex vault template organizes project records, collaborator notes, and reusable skills outside any single conversation. Agents have a place to consult before acting and to record approved updates afterward. Jason also keeps his vault in Git so he can review changes rather than reconstructing them from a long conversation history.

His distinction between scheduled tasks and scheduled messages asks whether the next run needs the earlier conversation. A daily summary can start fresh. A pull-request monitor benefits from remembering addressed comments, failed checks, and its stopping condition. Persistent agent threads allow the next check to continue that work. Jason describes improved context compaction as a reason he now keeps project threads running instead of routinely starting new conversations.

He makes a similarly practical distinction in computer use: native application control works across apps, Chrome supplies an authenticated browser session, and the in-app browser supports page and artifact review. He favors the narrowest tool that can finish the task. His published workflows require approval before sending, publishing, purchasing, or submitting, and he warns that an agent may switch interfaces to perform an action a connector restricts.

His enthusiasm for delegation has limits. For personal skills, Jason often starts quickly and corrects mistakes through use; he gives shared team skills more scrutiny. He acknowledges that evaluating workflows connected to changing Slack and other application state is difficult, and that some elaborate monitoring arrangements may not justify their token costs. He encourages lower reasoning effort for routine tasks rather than assuming that more thinking always produces a better result.

Practicing attention

Delegation changes what Jason wants to do with his own time. He encourages skills that benefit teammates and conversations with colleagues that reveal context an agent cannot gather alone.

His writing on taste explores the cost of making production easier. In art school, he could recognize work he lacked the ability and vocabulary to produce. He argues that AI can reverse that imbalance: people can make more than they know how to judge. Transcribing a musical solo develops an ear that receiving a finished chart does not; trying on clothes reveals material and shape in ways photographs cannot. Drawing on music, clothing, and pottery, Jason recommends experiencing work closely, noticing the decisions behind it, and learning to explain why something succeeds. His concern is that tools can remove the very attention through which taste develops.

Talks by Jason Liu

5 talks

Loading more talks and highlights…

References