How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase
Read the talk
How to Know When an AI Agent Can Work Unsupervised
Evan Kormos presents a practical architecture for regulated workflows: deterministic policy and process boundaries outside, adaptive agent loops inside, and measured confidence gates that reduce human review only when the evidence supports it.
From a talk by Evan Kormos
At a glance
Ideas worth remembering
Model the regulated outer process as a deterministic DAG, then allow bounded adaptive loops inside selected nodes.
A loop deserves no more trust than its acceptance check; repeated passes do not guarantee improvement.
Use deterministic tests for conventional software behavior, repeated evals for variable outputs, and traces to inspect how agent runs actually unfolded.
Automate nodes that are easy to check against policy, bounded, traceable, and able to fall back to a human.
Reduce oversight progressively—from full review to sampling to exception handling—only after measured confidence clears organization-defined policy thresholds.
Monitor the whole workflow: speeding up one node can move volume into design, security, or another downstream bottleneck.
Policy is the original system prompt
Evan Kormos, a player-coach at Coinbase with fifteen years in enterprise financial services, starts agent autonomy somewhere deliberately boring: policy. In a regulated organization, policy records institutional intent using words such as must, may, and should. It defines how work is allowed to proceed, especially where risk and compliance matter. In that sense, policy is the organization’s original system prompt—but unlike an ordinary prompt, it predates and governs the running software around it. 1:00
Policy and execution occupy different time scales. The organization periodically defines or changes policy—the build state—while products, transactions, and operational work continue moving through the run state. Product changes can make the running system evolve quickly, but its actions are still expected to remain inside the policy boundary. Separating these states matters because an agent cannot be judged only by whether its latest output looks plausible; it must also remain consistent with rules established outside the current run. 1:32
Agents then cut across the familiar enterprise trio of people, process, and technology. They collaborate with people, execute parts of a process, and depend on technology. The useful design question is therefore narrower than “Where should we use AI?” It is: Which process steps should an agent perform, what boundaries should constrain them, and what evidence should make their outputs trustworthy?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Map the outer workflow as a policy-bounded DAG
A regulated workflow has direction and gates. A software change may require approval before release; a customer may require verification before an account opens. Mapping those dependencies makes the rules inspectable: one step cannot begin until its prerequisites finish, and required checks cannot quietly disappear when an agent enters the process. 3:12
Agents make it economical to revisit which work truly needs to be sequential. Independent reviews that were once too expensive or difficult to coordinate can run alongside one another, after which their results converge. Parallelism changes capacity, not policy: the required checks remain in place even when independent work happens simultaneously.
This outer structure is a directed acyclic graph, or DAG: nodes represent steps, edges represent dependencies, and execution moves forward without returning to an earlier node. Apache Airflow is one example of software that defines such dependencies in Python. The same representation can describe commercial lending, regional token listing, or regulated software delivery. The value lies less in the graph vocabulary than in making ordering, parallel work, and gates explicit. 4:05
Parallel work inside a gated workflow
Read the diagram as text
- Work enters the process. A change or customer case begins a policy-driven workflow.
- Independent review A. parallel — One review can proceed without waiting for the other.
- Independent review B. parallel — Additional capacity runs alongside the first review.
- Required gate. Results come together and required checks remain enforced.
- Next workflow step. Approved work moves forward without looping back through the outer DAG.
- Work enters the process → Independent review A: dispatch.
- Work enters the process → Independent review B: dispatch.
- Independent review A → Required gate: result.
- Independent review B → Required gate: result.
- Required gate → Next workflow step: approved.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Inside each node, the work becomes an adaptive loop
Zoom into a single DAG node—code review, for example—and the shape changes. Work is produced, checked, and either accepted or returned with feedback. A satisfactory change advances to the next process step. An unsatisfactory change uses the review feedback to iterate until it becomes acceptable. This inner structure is cyclic even though the outer workflow remains acyclic. 5:37
Kormos calls this a directed cyclic graph and points to LangGraph-style state management as useful for such loops. The label matters mainly because it distinguishes two responsibilities: the outer DAG preserves policy-driven ordering, while the inner loop lets people and agents adapt their work before it advances. Real graphs can be much more complex than this simplified code-review example. 6:36
The process also needs exception paths. A release can be canceled, fail a security standard, or encounter a case too unusual to encode in advance. These exceptions do not necessarily originate in the acceptance check itself: cancellation, for example, can interrupt work independently. Designing every edge case is impractical, but letting unexpected work stall silently is worse. Exception handling should make these cases visible and route them for intervention rather than pretending the normal path still applies.
Automation then moves the bottleneck. Faster code review increases pressure on design, security scanning, or whatever step follows. Parallelism can create several new queues at once. That is why end-to-end operational metrics matter more than the elegance of the workflow-authoring tool: without visibility into volume and waiting time across the whole process, a local speedup can merely relocate delay. Additional capacity should be added where the flow shows pressure, while controls remain intact. 7:43
The check controls the inner loop
Read the diagram as text
- Produce or revise work. A person, an agent, or both perform the node’s task.
- Check the result. Determine whether the work is good enough to continue.
- Advance to the next node. normal — Acceptable work leaves the inner loop and re-enters the outer DAG.
- Exception handling. exception — Canceled, noncompliant, or unusual cases leave the normal path for intervention.
- Produce or revise work → Check the result: submit.
- Check the result → Produce or revise work: feedback: revise.
- Check the result → Advance to the next node: good enough.
- Produce or revise work → Exception handling: canceled or diverted.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The loop is only as good as the check
Repeated work-check-revise cycles sound straightforward, but AI makes both inputs and outputs variable. Better context can improve results and reduce wasted compute, although the talk does not quantify that relationship. One practical input is the current DAG state: tell the agent which node it occupies, what preceded it, and what the larger process is trying to achieve. That gives the local loop enough context to serve a longer-horizon goal. 9:14
Longer horizons make inefficient loops expensive. Every pass consumes time and compute, so repetition is useful only if the check can distinguish improvement from motion. The system must define what “good” means and evaluate it reliably. Otherwise, another pass may change the output without making it better.
This is the talk’s sharpest rule: “The loop is really only as trustworthy as the check.” Agent capability does not rescue a vague acceptance criterion. If policy cannot be translated into a reliable check for a node, that node is a poor candidate for unsupervised operation no matter how polished individual outputs appear.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evals and traces complement deterministic tests
Traditional software already has confidence mechanisms. Unit tests and other test cases run as part of delivery, checking that a change preserves expected behavior and does not break existing functionality. These checks work well when the same input should produce an exact or otherwise deterministic result. 11:36
Agents complicate that model because the same request can produce different answers, and several answers may be acceptable. Exact output comparison therefore cannot carry the whole quality argument. The replacement is not one magical score but a representative dataset, explicit definitions of acceptable behavior, repeated evaluation runs, and analysis over a distribution of outputs rather than one favored example.
Evals estimate how often outputs satisfy those criteria across representative cases. Traces record what happened during execution: the steps, model activity, and tool behavior that produced an output. Together they form a confidence mechanism complementary to conventional tests. Tests still protect deterministic software behavior; evals measure variable outcomes; traces make failures diagnosable and support monitoring when humans no longer inspect every run. 12:38
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Earn autonomy one node at a time
The lending example turns the architecture into a selection test. KYC and financial analysis may run in parallel, followed by steps such as a credit memo, risk review, documentation, and a decision. Each node should be assessed independently rather than declaring the entire lending workflow “agentic.” Agent assistance may be useful throughout the DAG even when only some nodes are candidates for eventual takeover. 13:12
Four questions identify the better candidates:
- Policy checkability: Can the output be compared with policy without relying on an undefined feeling of quality?
- Bounded execution: Can the agent work inside a loop with clear limits rather than continue indefinitely?
- Traceability: Can the organization inspect how the result was produced?
- Human fallback: Can an unresolved or risky case be handed to a person?
A node that answers yes to these questions is better suited for automation. A final lending decision remains harder because it is itself a consequential check against policy and may need to stay human-driven.
Autonomy then increases through three operating modes. Fully supervised: humans manually verify every agent output while domain experts define good outcomes and the team improves datasets and evaluators. Sampled: after measured confidence reaches a policy threshold, humans inspect only a slice, while traces and systematic evaluation cover the unseen remainder. Exceptions only: after another confidence gate, humans concentrate on unusual cases while continuing to refine datasets and quality criteria.
The progression is a governance design, not a numerical recipe. The recording does not specify the confidence metric, threshold values, sampling rate, evaluation set size, or evidence needed to establish statistical reliability. Those must come from the organization’s policy, domain experts, risk tolerance, and observed failure costs. Without that specification, “measured confidence” is still too vague to authorize unsupervised work.
The resulting recipe is compact: keep the outer process deterministic and policy-bound; let the inside of selected nodes remain adaptive; and use evaluation evidence rather than confidence in a model’s reputation. Quality remains a team responsibility because domain experts define acceptable outcomes, AI engineers build the evaluation and trace pipeline, and operators monitor how acceleration changes the entire workflow. Agents can accelerate the operating model, but autonomy should expand only as fast as the checks can justify it. 15:49
Oversight decreases only after evidence clears policy gates
Read the diagram as text
- Fully supervised. Humans verify every output; domain experts refine criteria and datasets.
- Policy confidence threshold. gate — Measured performance must justify reducing review coverage.
- Sampled review. Humans inspect a slice; traces and systematic evals cover the remaining output.
- Confidence threshold cleared. gate — Evidence must justify exception-focused operation.
- Exceptions only. Humans handle unusual cases and continue improving datasets and quality definitions.
- Fully supervised → Policy confidence threshold: collect evidence.
- Policy confidence threshold → Sampled review: threshold reached.
- Sampled review → Confidence threshold cleared: continue evaluation.
- Confidence threshold cleared → Exceptions only: threshold cleared.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Related talks
- 3 ingredients for building reliable enterprise agents
Extends the same hybrid architecture: valuable agent work inside deterministic workflows, with observability, evaluation, reversibility, and human correction.
- Evals Are Not Unit Tests
Develops the distinction between deterministic software tests and evaluations for nondeterministic model behavior using representative production inputs.
- Building Deterministic Infrastructure for Non-Deterministic AI Agents
Explores the surrounding distributed-systems controls needed to operate stochastic agents with persistent state, observability, cost limits, and human oversight.
Full transcript
Loading transcript…