AI Engineer Summit 2025

AI Agents, Meet Test Driven Development

Loading slides…

Read the talk

AI Agents, Meet Test-Driven Development

Reliable agents need more than capable models: representative tests, observable decisions, and bounded feedback loops turn promising examples into workflows that can improve in production.

From a talk by Anita Kirkovska

Before you start: Familiarity with LLM prompts, API calls, and basic software testing is helpful; the article introduces retrieval, model judges, and agentic workflow levels as they appear.

What makes an AI wrapper work in production?

What turns an application wrapped around an AI model into a durable product? Anita Kirkovska approaches that question through her work leading GenAI Growth and Education at Vellum. She reports that, across hundreds of companies the team worked with, those adopting test-driven development built stronger, more reliable production systems. The contrast starts with the skepticism of 2023, when AI wrappers were widely dismissed as lacking defensibility. Kirkovska reports that Cursor subsequently reached $100 million in annual recurring revenue in 12 months, describing it as the fastest-growing SaaS business. That is her framing of the market’s reversal, rather than an independently established growth comparison.

Opening slide with Cursor annual recurring revenue and a comparison of years to reach $100 million ARR.Open full source frame
Kirkovska opens with Cursor’s reported revenue growth before turning to what makes an AI application useful.

Better coding models helped. So did broader AI adoption and the suitability of programming as an early application. But another change happened around the models: developers learned to orchestrate calls, connect models to application data, and make the resulting systems operate effectively in production. Model capability and application engineering advance together. A stronger model does not remove the need to organize its work.

That surrounding engineering matters because hallucinations, overfitting, and inconsistent output structure remain practical problems. Kirkovska describes the apparent slowdown after the leap from GPT-3.5 to GPT-4, then questions whether it represents a real ceiling. Scaling training data and model size was no longer the only avenue worth considering.

0:000:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Better reasoning still needs a surrounding system

Reinforcement learning and additional computation before answering opened another path. The talk uses DeepSeek-R1 to illustrate this shift, but its description of training without labeled data needs a distinction: the original DeepSeek paper describes DeepSeek-R1-Zero as using reinforcement learning without preliminary supervised fine-tuning, whereas R1 incorporates cold-start data and multiple training stages. Neither description means learning without pretraining data.

Kirkovska connects that development to OpenAI’s o1 and o3 reasoning models. OpenAI’s account of training o1 supports the general connection between reinforcement learning, chain-of-thought reasoning, and additional inference computation; it does not establish an identical training recipe for all these models. The application-level consequence is straightforward: spending more computation before answering can help a model tackle more complex reasoning tasks.

Tool use, research capabilities, and document understanding also expand what workflows can do. Kirkovska characterizes Gemini 2.0 Flash as having near-perfect OCR accuracy, without specifying a dataset or scoring conditions. That makes it a candidate to test on the documents a product actually handles, not a universal OCR guarantee. Meanwhile, Humanity’s Last Exam illustrates why harder benchmarks emerged as older ones became saturated: even capable contemporary models struggled with its difficult questions. Better models expand the available building blocks, but production success still depends on how those blocks are assembled.

The application toolkit developed in parallel. Chain-of-thought prompting encouraged more explicit reasoning; retrieval-augmented generation, or RAG, grounded answers in an organization’s data. Memory retained context across conversations, while longer context windows opened additional use cases. GraphRAG brought graph structure and hierarchy into retrieval. Reasoning models added more time spent thinking at response time, and agentic RAG introduced decisions about retrieval into the workflow itself. The engineering task is to choose the combination of techniques, models, and logic that fits a well-understood problem—not to accumulate every available technique.

2:312:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:31 · section reference included

Establish feasibility before committing engineering time

The development cycle starts with experimentation, moves into evaluation at scale, and continues after deployment through monitoring and improvement. In this setting, test-driven development is a continuing feedback process: demonstrate that a workflow can solve the problem, test it against a broader range of cases, then use production behavior to improve the next version. A successful prototype is the beginning of that process.

During experimentation, compare approaches that change how the work is divided:

  • Few-shot prompting: Supply examples that show the model the desired task and response pattern.
  • Chain-of-thought prompting: Explore whether a reasoning-oriented prompt helps on more complex tasks.
  • Prompt chaining: Split a large instruction set across multiple prompts, giving each stage a narrower job.
  • ReAct: Interleave reasoning and actions so feedback from the environment can inform the next step. The ReAct paper describes this interaction more specifically than a single planning stage followed by an answer.

These are alternatives to evaluate against the task, rather than a progression every workflow must follow.

Domain experts belong in this phase because they know what a useful answer looks like and which omissions matter. Kirkovska argues against making engineers solely responsible for prompt tuning: expert participation can establish feasibility before the team commits substantial implementation time. Remain model-agnostic, too. Compare models on the actual job they must perform; Vellum’s favorable experience with Gemini 2.0 Flash for OCR is her example of matching a model to a particular task.

5:365:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:36 · section reference included

Move beyond a handful of successful examples

A few good outputs establish possibility, not production coverage. Kirkovska asks what happens when a system faces hundreds, thousands, or even millions of requests per minute. Those are hypothetical demand levels, not measured throughput. Her next step is to assemble an evaluation dataset containing hundreds of examples and use it to expose the compromises that a small demonstration can hide.

Evaluation slide listing constraints, ground-truth data, custom metrics, a flexible test bank and offline, inline and online evaluations.Open full source frame
The evaluation checklist connects quality, cost, latency and privacy to ground truth, custom metrics and offline, inline and online tests.

Define priorities before selecting the final model and workflow:

PriorityDecision it influences
QualityWhether slower, more deliberative responses are acceptable
CostWhether a lighter, cheaper model is sufficient
LatencyHow much processing the response can afford
PrivacyWhich data-handling choices are acceptable

No configuration optimizes every dimension simultaneously. A high-quality result may justify a longer wait; a cost-sensitive task may require a smaller model. Making those priorities explicit lets the evaluation answer a product question rather than merely produce a score.

Use ground-truth examples where possible, with subject-matter experts helping build the dataset and assess the expected behavior. Generic or synthetic benchmarks can be useful, but they do not necessarily represent the product’s inputs or acceptance criteria. Without ground truth, an LLM judge can evaluate another model’s response. Kirkovska presents this as a standard, reliable option; in practice, the judge itself needs scrutiny. Research on LLM judges documents position, verbosity, self-enhancement, and reasoning biases, so an editorial or correctness rubric should be calibrated against expert judgment.

The testing framework must accommodate changing workflows and nondeterministic responses, whether it is built internally or supplied by a service. It should retain varying outputs and allow custom metrics in Python or TypeScript. Evaluation also belongs inside the workflow: guardrails at intermediate nodes can check whether each stage produced a correct result before later stages depend on it. Run those checks while prototyping, then revisit them when production supplies real examples. A final answer alone cannot reveal which internal decision failed.

7:357:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:35 · section reference included

Deploy a system you can observe and change

Once expert review and evaluation justify deployment, the work shifts toward understanding actual behavior. Record LLM calls, their inputs and outputs, and latency so failures can be traced through the system. This becomes especially important when an agent can choose different branches: debugging requires knowing both what it returned and which path it took to get there.

Operational reliability also depends on the services underneath the workflow. Use retries and fallback logic to handle failed API calls and provider outages. Kirkovska cites an OpenAI outage lasting four hours, which she places two months before the talk, as an example of why a workflow might need to switch to another model. The recording does not identify the incident or affected endpoints.

Version prompts and workflow definitions, and test changes in staging before a wider rollout. A prompt improvement in one part of a system can introduce a regression elsewhere. AI-feature deployments should also be separable from the application’s normal release schedule: prompts and workflow logic may need adjustment more frequently than the rest of the product.

Production then becomes a source of new test cases. Capture user interactions and edge cases, add them to evaluation, and check whether revised prompts fix the newly observed behavior. For repeat queries, caching avoids paying for the same expensive model call repeatedly and reduces response time by serving a stored result.

After the product has operated reliably for a period the team is comfortable with, accumulated data may support fine-tuning a custom model. Kirkovska proposes this as a later optimization for more task-specific responses, reduced dependence on external API calls, and potentially lower costs. The sequence matters: collect experience from a working system before using that experience to specialize its model.

10:1310:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:13 · section reference included

Evaluate the decisions an agent is allowed to make

An agentic workflow may call multiple APIs, use many tools, or coordinate agents executing in parallel. Testing each output is therefore insufficient: evaluation must also ask whether the system made the right decisions and followed the intended logic. Kirkovska organizes that problem around control, reasoning, and autonomy. Her provisional framework runs from L0 through L4; it is a way to discuss behavior, not a fixed definition of an agent.

L0 keeps orchestration fixed. A workflow calls an LLM, retrieves information from a vector database, may run inline evaluations, and returns a response. Reasoning can occur inside the model, but there is no external agent organizing a plan or deciding which actions to take. The distinction is between reasoning within a prompted call and control over the surrounding workflow.

L1 gives the model tool decisions. The system chooses when to call an API, invoke another tool, or retrieve additional information before answering. Memory retains context across conversations and potentially parallel activity. Evaluation must cover the chosen tool, the decision to use it, and the accuracy of the resulting answer. L1 can already be complicated: Kirkovska imagines ten available tools, with an agent choosing whether it needs the first five or the last two. A large branching graph does not, by itself, imply a higher level of autonomy.

L2 adds structured planning and refinement. A trigger can start a process that decomposes a task, retrieves information, calls a tool, evaluates whether the result is useful, and revises the next step. The system is deciding what needs to be done, rather than only whether to call a listed tool. Its reasoning loop is still finite: after completing the planned work and producing an output, it terminates.

L3 stays active. Instead of responding to one request and stopping, the system continuously monitors its environment and acts when needed. With access to email, Slack, Google Drive, or other services, it can plan a next move, execute an action, or ask a person for input. Kirkovska’s example is a marketing assistant that prepares a video or presentation for someone to use later. Persistence and proactive behavior distinguish it from a finite task runner.

L4 invents new workflows. It would create the agents, prompts, function calls, tools, or utilities needed to solve a problem in a new way. Kirkovska treats true L4 as aspirational at the time of the recording, citing overfitting and inductive biases tied to training data as constraints. The proposed change is from operating within an existing process to creating the process itself.

13:4314:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:43 · section reference included

Useful production systems need not maximize autonomy

Kirkovska places Vellum customers Redfin, Drata, and Headspace in L1. Her emphasis is on orchestration: making models interact correctly with application systems and retrieving context that actually matches the user’s question. A tool-using workflow can be simple or elaborate, but its value depends on the quality of those interactions. Experiments across modalities, tools, and models need continuous testing to determine whether a change improves the system.

Looking ahead from the recording, she expects much of that year’s innovation at L2, with models such as o1, o3, and DeepSeek supporting planning and reasoning over complex tasks. That also creates room for new interfaces: users need ways to work with agents carrying out longer sequences of activity. L3 and L4 remain constrained in her account by both models and surrounding logic, even as experimentation continues.

Before moving into the demonstration, Kirkovska points to a companion resource covering agent architectures, places to test them, and lessons from researchers and practitioners. The practical example then brings those ideas together in an SEO workflow that combines tool use, research, writing, and editorial evaluation.

20:4020:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

20:40 · section reference included

Build research and evaluation into the writing process

The SEO agent automates keyword research, content analysis, and content creation. It decides when to use tools and includes an evaluator that acts as an editor. Kirkovska places it between L1 and L2: it combines tool selection with a structured process for developing and refining a draft.

The workflow separates responsibilities so each stage supplies something concrete to the next:

  1. Analyze competing content. An SEO analyst and researcher take a keyword, use Google search, and examine top-performing articles.
  2. Identify strengths and gaps. Preserve useful coverage from those articles while identifying missing topics or areas that deserve better treatment.
  3. Write from the collected context. Give the writer the research and planning material needed to produce a first draft.
  4. Evaluate the draft. An editor implemented as an LLM judge applies predefined rules from its prompt.
  5. Revise with memory. Return the editor’s feedback to the writer and retain their conversation history across the loop until a stopping criterion is met.

The intended result is a substantial first draft Kirkovska can develop further. Competitive research and an editorial loop do not establish that the article will outrank its sources; they give the writer better context and a mechanism for revision.

23:0623:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

23:06 · section reference included

Follow “chain-of-thought prompting” through the workflow

The live run uses the keyword chain-of-thought prompting, along with writing-style and target-audience inputs. The SEO analyst examines the articles Google ranks for that keyword. It identifies useful components to retain and missing opportunities for additional coverage. The researcher then performs another search to fill those gaps, rather than simply repeating the analyst’s original search.

The writer receives the accumulated material as context for its first draft. Kirkovska also describes an optional RAG connection to an internal database of articles and learnings. That would add the organization’s own material to the external research; it is an extension she proposes, not a step she establishes as part of this run.

After the first draft, the editor supplies feedback. That feedback passes through a memory component holding the writer–editor chat history and then through a node that structures the writer’s next input. The loop has two ways to stop: an excellent assessment from the editor, which Kirkovska says is rare, or the demonstration’s limit of one revision cycle. In the run, the cycle limit ends the process even though the editor still has feedback. Stopping and passing evaluation are different outcomes.

The same stopping policy can be expressed directly in Python. Here, revision_count counts completed writer–editor revision cycles after the initial draft. The keyword is unchanged from the demonstration, and stop_reason distinguishes an accepted draft from one returned because the revision budget is exhausted:

python

from dataclasses import dataclass
from typing import Literal


@dataclass(frozen=True)
class LoopDecision:
    keyword: str
    revision_count: int
    stop_reason: Literal["excellent", "revision_limit"] | None


def decide_next_step(
    keyword: str,
    revision_count: int,
    editor_says_excellent: bool,
) -> LoopDecision:
    if revision_count < 0:
        raise ValueError("revision_count must be nonnegative")

    if editor_says_excellent:
        reason = "excellent"
    elif revision_count >= 1:
        reason = "revision_limit"
    else:
        reason = None

    return LoopDecision(keyword, revision_count, reason)


keyword = "chain-of-thought prompting"

# Initial editorial feedback: another writing pass is allowed.
initial = decide_next_step(keyword, 0, False)
assert initial.stop_reason is None

# Feedback remains after one revision, but the demo budget is spent.
revised = decide_next_step(keyword, 1, False)
assert revised.stop_reason == "revision_limit"

This isolates a deterministic part of an otherwise model-driven workflow: whether another iteration is permitted. The editorial judgment and the loop-control decision can be tested separately.

The resulting article is titled “mastering chain-of-thought prompting in AI, a comprehensive guide for developers”. Kirkovska considers changing the title, but sees the article’s coverage as a useful foundation because it reflects the material gathered during competitive research. She reports a demo latency of around 118, without specifying a unit. Kirkovska reports that the SEO workflow usually takes around 300 seconds when it runs more evaluation loops. The practical result is a draft she can build on, with the additional editorial work traded against a longer run.

25:0425:13
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

25:04 · section reference included

Keep workflow definitions in the codebase

The demonstration runs in Vellum Workflows, which Kirkovska describes as a way for product and engineering teams to collaborate on tested AI development. The closing announcement addresses a complementary need: developers want control, flexibility, and ownership of workflow definitions in their codebase. The Workflows SDK supplies customizable building blocks and syntax intended to make an agent’s structure and behavior legible directly in code.

Kirkovska presents synchronization between the UI and code as the bridge between those modes of work: teams can define, debug, and improve the same workflow through either representation. She announces the SDK as free and open source. The open-source workflow library is distinct from hosted UI integration and certain nodes, which require a Vellum account. The substantive endpoint is shared ownership of an inspectable workflow—domain experts can shape its behavior while developers retain its definition, control flow, and tests in code.

27:4928:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

27:49 · section reference included

Resources

From the talk

  • Python SDK for defining AI workflows in code, with examples and integration with Vellum's visual editor. Some features require a Vellum account.

  • Original paper introducing a difficult benchmark spanning academic subjects.

  • Introduces a method that interleaves language-model reasoning with actions and feedback from the environment.

  • Documentation for Microsoft's graph-based approach to retrieving and summarizing information from a text corpus.

  • Historical release notes describing SDK-enabled workflows and pulling workflow definitions into code, editing them, and pushing them back to the UI.

  • Explains the distinct training approaches behind DeepSeek-R1-Zero and DeepSeek-R1.

  • OpenAI's original explanation of reinforcement learning and inference-time reasoning in o1.

  • Research on model-based evaluation, agreement with human judgments, and recurring judge biases.

Transcript download
Full transcript

Loading transcript…