Contents
  1. Purpose and development
    1. Define the completed work
    2. Turning points in process automation
  2. Redesign and allocation
    1. Change the process before automating it
      1. What event records reveal
      2. Change dependencies deliberately
    2. Allocate work and authority separately
      1. Make rule conflicts visible
    3. Bound the dynamic decision
  3. State and coordination
    1. Name the authoritative records
    2. Accept events against current state
    3. Transfer responsibility explicitly
      1. Coordinate parallel obligations
  4. Human decisions and exceptions
    1. Make review a consequential decision
      1. Test the reviewer decision
    2. Operate the exception workload
      1. Capacity constrains automation
  5. Integration and recovery
    1. Preserve meaning across applications
    2. Recover from the effects that occurred
      1. Partial completion is still real
  6. Outcomes and daily operation
    1. Measure the complete process
      1. Interpret the comparison
    2. Keep the workflow usable and owned
  7. Check understanding
  8. Open questions
  9. Selected talks
  10. References
  11. Talk library
← All topics

Workflow automation

Workflow automation coordinates work across people and applications. Its purpose is to make a useful result easier to obtain—not simply to make individual tasks run faster. The central design decision is which work to remove, which to implement with explicit rules, where learned interpretation helps, and where people should retain judgment or authority. Those choices become a dependable process only when handoffs, exceptions, recovery and operating responsibility are designed with them.

Purpose and development

Define the completed work

A business process connects activities that transform inputs into a result. Its workflow specifies how that work proceeds: which activities occur, what permits advancement and how people or software participate. Automating an activity changes how one part is performed; redesigning the workflow can also change which activities are needed and how they depend on one another.

A case is one instance of the work: one service request, invoice discrepancy or application. The reusable process describes how cases should be handled; a particular case has its own supporting records, completed work and remaining obligations. Changing the process definition and advancing that case are different operations. The Case Management Model and Notation specification formalizes this distinction between a case model and its instances.

Define completion from the recipient's perspective. Drafting a support reply produces an artifact; resolving the request requires satisfying the agreed service need. A disposition records how the case was handled, such as fulfillment, rejection or cancellation. These can all be legitimate outcomes, but they are not interchangeable. Even a closed status need not establish customer-confirmed resolution: Zendesk can close tickets through automation, after which a reply creates follow-up work rather than reopening the original.

Someone must maintain the complete arrangement, including obligations that cross teams. This process owner defines and maintains the process, as described in the ISO quality-management vocabulary; ownership does not automatically grant authority over every participating system or transaction. Before selecting automation, agree on the initiating condition, case identity, intended recipient, required result, allowed dispositions and downstream obligations. The customer acceptance agreement establishes these requirements; workflow design makes them operational.

Turning points in process automation

Process automation draws on several traditions because improving flow, changing the work itself, coordinating execution and allocating human responsibility are different problems. Their historical development explains why a more capable model does not make the older concerns disappear.

DevelopmentContribution
Toyota's stopping and flow traditions — 1896 and 1938 antecedentsToyota traces automatic stopping to Sakichi Toyoda's 1896 power loom and flow production at its Koromo plant to Kiichiro Toyoda in 1938. In the later Toyota Production System, jidoka means stopping on abnormality; Just-in-Time coordinates production with what is needed, when and in what quantity. These address quality and whole-process flow, not merely machine speed.
Business process redesign — June 1990Thomas Davenport and James Short's The New Industrial Engineering treated information technology and process redesign as mutually informing choices. Their cross-organizational perspective challenged departments optimizing their own tasks while nobody owned the complete result.
Workflow Reference Model — 29 November 1994David Hollingsworth and the Workflow Management Coalition's archived reference model separated process definition, execution control and interaction with people and applications. Common components and interfaces addressed incompatible workflow products. It explicitly distinguished redesigning a procedure from automating it unchanged.
Function allocation — May 2000Raja Parasuraman, Thomas Sheridan and Christopher Wickens's human–automation framework separated acquiring information, analyzing it, choosing actions and implementing them. Different functions could receive different degrees of automation.

Business process reengineering concerns redesigning the process itself, rather than simply accelerating its existing tasks. Business process management is the broader continuing work of defining, operating and improving processes. Neither requires every activity to be automated. Manufacturing flow principles also require adaptation to service work: a waiting request represents an obligation to someone, not inventory that can necessarily be discarded. These traditions supply complementary design questions, not a ladder toward eliminating people.

Redesign and allocation

Change the process before automating it

Start from observed work, not just the procedure manual. The perceived process, actual process and desired process can differ. Map activities, responsible roles, decision conditions, waiting, variants and correction. Learning from work as it happens explains discovery; here its findings become decisions about what to retain, remove or rearrange. AHRQ's workflow-mapping guidance recommends checking descriptions through observation and testing redesigns before expanding them.

A bottleneck is the constraint limiting flow through the complete process. It is not necessarily the task with the longest individual duration. A quick approval performed by one overloaded reviewer can hold up more work than a lengthy activity with ample capacity. Separate active work from waiting, then identify what prevents useful completions. Faster drafting can otherwise move the constraint into review. Maggie Appleton's collaborative-engineering account describes how alignment deferred until a pull request leaves teams reviewing already-built work whose purpose was never agreed.

What event records reveal

Process mining uses recorded events to investigate how work actually proceeds. Each event identifies an activity and its case; timestamps help reconstruct timing, while records of who or what performed the activity help locate responsibility. Grouping events by case reveals recorded paths. Choosing an order as the case gives a different view from choosing an order line or delivery. Wil van der Aalst and collaborators' Process Mining Manifesto, developed for the 2011 Business Process Management workshops and published in 2012 proceedings, distinguishes three uses: discovering a process model from events, checking recorded behavior against an existing model, and enhancing a model with event information. None recovers work that was never recorded. Missing phone calls, informal approvals and off-system correction therefore require observation alongside log analysis.

Change dependencies deliberately

Redesign in this order: remove work that serves no requirement, simplify retained work, improve inputs and reconsider dependencies. Consider a small example in which a coordinator copies a submitted request into a second register, obtains approval and only then checks availability. Suppose the register adds no required information and the availability check is read-only and independent of approval. The revised process can reuse the original record and perform that check earlier, while fulfillment still requires both approval and availability. Removing the copy and changing the dependency are separate decisions; neither relaxes the completion requirement.

Change dependencies, preserve obligations

Change dependencies, preserve obligationsThe old path serializes request, duplicate entry, approval, availability check and fulfillment. The revised path removes copying, forks the original request into approval and read-only availability, then requires both valid results before fulfillment.Before · copied entry and serialized prerequisitesRequestRequesterDuplicate entryCoordinatorApprovalApproverAvailability checkOperations · read-onlyFulfillOperationsAfter · original record reused, both obligations retainedRequestRequesterApprovalApproverAvailability checkOperations · read-onlyBoth holdExplicit joinFulfillOperationsValid approvalAcceptable availabilityOnly copying disappears. Approval is still required; a completed rejection does not satisfy the join.
This fictional request process removes duplicate entry and an unnecessary dependency. Approval and the read-only availability check can proceed independently; fulfillment still requires valid approval and acceptable availability. Edges express prerequisites, not elapsed time or measured savings.

The proposal must still be checked with the people operating dependent processes. Earlier checks might become stale before fulfillment; a supposedly duplicate entry might support another team's obligation. The test is whether the changed arrangement preserves those obligations with less unnecessary work, not whether its diagram contains fewer boxes.

Allocate work and authority separately

Function allocation assigns parts of the work to people or software. Parasuraman, Sheridan and Wickens's framework is useful because collecting information, interpreting it, selecting an action and executing it need not share an executor. Their proposed evaluation considers human performance, automation reliability and the consequences of incorrect actions. Automating collection can help while leaving a consequential choice with a person; automating execution can also be appropriate after another actor has settled that choice.

Ordinary software applies explicitly programmed rules. A model applies behavior fitted from examples, which can help interpret varied language or documents; Machine Learning Fundamentals develops that distinction. Interpretation does not confer authority. A model can propose a request category, deterministic software can check explicit eligibility conditions, and a person can resolve disputed intent. Keep mandatory conditions and validated business state outside the model's discretion.

Choose an arrangement from the properties of the work, not from an autonomy ranking.
Work propertyExecution choiceAcceptance and authority
No required result depends on the activityRemove itProcess owner confirms that no obligation is lost
Explicit rules and structured inputsDeterministic softwareValidate inputs and enforce the rule at the protected operation
Variable wording, but a checkable interpretationModel proposes structured informationValidate against sources; clarify material ambiguity before commitment
Next investigation depends on findingsBounded agent stageLimit available operations and effort; return findings to the decision owner
Disputed intent or unsettled tradeoffsHuman decision with software assistanceThe responsible person settles the choice; software still enforces applicable constraints
Rare work or costly maintenanceRetain manual executionCompare the full operating cost before automating

Input ambiguity and rule clarity determine what can be specified. Consequence, reversibility and verifiability determine how much independent checking or authorization is needed. Frequency and maintenance burden determine whether implementing the automation is worthwhile. Decision support assists a decision owner without taking over the decision. Here, straight-through processing means completing a case without human intervention; it describes a path through the process, not a quality guarantee.

Make rule conflicts visible

A decision table maps input conditions to outcomes. Overlap needs explicit semantics. Suppose example rules say “amount below 100 → routine” and “new supplier → review.” Both match a new supplier's amount of 50. In Decision Model and Notation, Unique prohibits overlapping rules; Any permits overlap only when outputs agree; First chooses the first match. First therefore makes row order consequential and can hide inconsistencies. A table makes policy executable, but cannot establish that the policy itself is appropriate.

Bound the dynamic decision

A prescribed workflow can contain branches, repetition and parallel operations. An agent adds discretion over subsequent actions within a delegated task. Always reading three named records is a prescribed procedure; choosing which record to inspect next from an unexpected finding introduces a dynamic decision. Agent Engineering explains this boundary. Start with the simplest sufficient arrangement: more discretion adds behavior to assess and can add latency and cost.

Use an agent where the investigation genuinely benefits from that choice, not merely because a model appears somewhere in the process. Michael Albada's Building Applications with AI Agents recommends simple chains and branches where sufficient, while treating difficult-to-maintain branching structures as a reason to consider fuller agency. This is an architectural tradeoff, not a universal threshold or measured guarantee that an agent will be easier to maintain.

Give the delegated stage a workflow-facing contract:

  • Purpose — The result it must return and the evidence needed to accept that result.
  • Reach — Permitted records, operations and external effects; investigation permission need not include repair permission.
  • Limits — Time, attempts or other effort allowances enforced by the surrounding system.
  • Return conditions — Completion, insufficient evidence, exhausted limits or a needed action outside its authority, with a named receiving owner.

The surrounding workflow still owns case disposition and mandatory gates. Bloomberg's described semi-agentic architecture, for example, makes policy and factuality checks non-optional rather than letting the agent choose whether to invoke them. Mandatory invocation and check effectiveness remain separate properties. The broader problem of limiting cumulative effects belongs in Constrain accumulated effects.

State and coordination

Name the authoritative records

A system of record is the designated authority for particular business facts. Authority can be divided: one application owns an address while another owns an order's fulfillment status. IBM's operational-data-store guidance treats this designation as a business decision and requires explicit overwrite rules between systems. Recall the agreed entities and meanings from Making business meaning explicit before assigning their writers.

Keep three kinds of state separate. A model proposal records a suggested change. Workflow state records orchestration progress, such as waiting for review. Business-record state records the accepted facts in the owning application. All three can be durable while disagreeing: a saved proposal is still only a proposal, and an execution marked finished may have stopped before the business requirement was met. A conversation can help reconstruct intent without becoming the authority for a calendar event, ticket status or payment.

For each case, preserve a mapping among its business identifier, execution identifiers and external record identifiers. A new execution attempt need not mean a new business case. Assign each mutable field a permitted writer, and define which observations justify changing it. For example, the workflow can own “awaiting confirmation,” while only a response or fresh read from the receiving system establishes its actual record state. This mapping gives later notifications and recovery work something stable to refer to.

A delayed proposal must be accepted against relevant current state, not merely the copy used to prepare it. HTTP's If-Match mechanism can make an update conditional on the current representation's version. That protection covers the representation identified by its validator, not every related business fact. Preserve state at acceptance explains concurrency enforcement. Reading first and writing later is insufficient if another writer can invalidate the checked conditions in between.

In this fictional case, saved proposals, workflow progress and application-owned facts describe different state. A delayed v1 proposal cannot overwrite the current v2 representation. The protected operation must also enforce authority.

Accept events against current state

A trigger is a request, notification or scheduled occurrence that starts or resumes consideration of work. Event correlation associates it with the correct case. Receiving a trigger does not itself admit a new case or authorize an operation. Scheduled and webhook-triggered agents are therefore workflow participants even when nobody has an active chat session open.

A transition changes workflow state; a guard is a condition that must hold before the transition is permitted. The SCXML state-machine specification separates the triggering event, guard and target state. Apply that separation to business work: a received document may satisfy a missing-information condition, but only a still-open, eligible case may advance.

In this example, a notification reports that information is available. Its event ID identifies the notification; the case ID identifies the continuing work.
Incoming notificationCurrent case stateDisposition
First receipt of event E1Waiting for the reported informationRead the information; advance only if the guard now holds
E1 delivered againIts notification was already handledDo not treat it as a second business request
A different, late eventCase cancelled or otherwise terminalApply the explicit reconsideration policy; do not silently reopen

Delivery order is not necessarily business order. Stripe's webhook documentation, for example, describes duplicate and out-of-order delivery, and event objects that do not change when the underlying resource changes. A fresh resource read can therefore answer a different question from an old event payload. Deduplicating a notification also does not prove that its downstream write occurred exactly once.

An update produced by the automation can itself generate another notification. IBM's synchronization example uses update-origin information to recognize already synchronized records and avoid update echoes. That addresses a different problem from repeated delivery of one event. Define planned waits, information deadlines and explicit reopening conditions as business policies; expiry can lead to escalation, rejection or cancellation, but should not acquire an accidental meaning from a worker timeout.

Transfer responsibility explicitly

A handoff transfers responsibility together with the information, access and authority needed to continue. Its work item should name the case, requested result, receiving role, required records, acceptance deadline, escalation owner and return conditions. Sending an email only requests attention. Receiving that message, accepting ownership, starting work and completing it are separate events.

OASIS's WS-HumanTask specification of 17 August 2010 gives this distinction an explicit lifecycle. Ready work is available to potential owners; claiming it makes it Reserved; starting makes it InProgress. Completion remains separate. Missing ownership requires administrative action, and start or completion deadlines can trigger escalation. These task states organize responsibility but do not prevent unrelated software from writing external records.

Specify who remains accountable until ownership is accepted. An operational example is PagerDuty's incident lifecycle: acknowledgment records that a responder has claimed an unresolved issue; without acknowledgment, escalation continues. An acknowledgment timeout can resume escalation. A notification-delivery receipt is consequently weaker than accepted responsibility, and accepted responsibility is weaker than a resolved incident.

Delivery is not accepted responsibility

Delivery is not accepted responsibilitySending owner remains accountable through delivery until claim. Recipient owns the task from claim through completion; starting is a separate event. An alternative unclaimed deadline triggers escalation while work remains outstanding.Event order; spacing is not elapsed timeDispatchDeliveredClaimStartCompleteSending ownerRecipientAccountable until claimAccountable after claimReady · availableReservedInProgressAlternative: no claim by the acceptance deadlineDelivered, still unclaimedSending owner remains accountableEscalate outstanding workNot task completion; no automatic releaseDeadline expiresCompleting this work item may leave broader case obligations outstanding.
Example policy: the sending owner remains accountable until the recipient claims the work. An unclaimed deadline triggers escalation. Ready, Reserved and InProgress use WS-HumanTask meanings; the accountability policy is not a claim that WS-HumanTask and PagerDuty share an implementation. Task ownership does not fence external writers.

Coordinate parallel obligations

Swimlanes group work by responsible role. Business Process Model and Notation, or BPMN, provides more precise distinctions: pools represent participants, lanes categorize activities within a process, sequence flows express order and message flows represent communication between participants. Those distinctions prevent one arrow from ambiguously meaning both “must finish first” and “sends information to.”

Parallel obligations need a join condition. If fulfillment requires both approval and an availability check, finishing either alone is insufficient. SCXML similarly completes an enclosing parallel state only when all child regions reach final states. In a business workflow, additionally define which results count as acceptable: rejection or cancellation of one branch must not masquerade as successful fulfillment. Demonstrating operating ownership extends this responsibility transfer to the people who will operate the whole system.

Human decisions and exceptions

Make review a consequential decision

Human review is deliberate assessment before accepting a result or performing an action. Human-in-the-loop means a person participates at a defined point in the process, but that phrase alone leaves their job unclear. Review assesses a proposal; clarification supplies missing intent; authorization grants permission; takeover accepts responsibility for unresolved work. Request the intervention actually needed, as developed in Review before consequential commitment.

Place review where consequence, uncertainty, irreversibility or required authority justifies it. Give the reviewer the relevant source information, exact target and proposed change, applicable rule, unresolved issue and allowed dispositions. Approve, reject and request changes perform different work. A reviewer who cannot inspect the source, understand the domain or spend enough time is not an effective control merely because a button must be clicked.

Enforce approval on the execution path. The n8n workshop demonstrates a configured layer that intercepts the ordinary tool invocation before execution, rather than relying on the agent to choose a separate review tool. The distinction is architectural: required review must not be bypassable through another route to the same effect.

Bind authorization to the significant action data and its validity conditions. OWASP's transaction-authorization guidance requires a final server-side gate and invalidation when transaction data change. Applied here, approval of one recipient, payload and scope must not silently authorize another. A revised proposal may need new approval; a delayed proposal needs its relevant state and authorization rechecked. Recording an earlier yes-click is not enough.

Test the reviewer decision

Automation bias is inappropriate reliance on automated advice. Test whether reviewers challenge wrong assistance, not just how often they approve it. In When Machines Mislead, researchers inserted fabricated cheating alerts into previously certified test sessions. Revised guidance requiring independent suspicious behavior was associated with model-estimated rejection of fabricated alerts rising from 50% to 71%; rejection of genuine operational alerts also rose. Those genuine alerts were not ground-truth misconduct, so rejecting them cannot automatically be counted as an error. The studies also used different sessions and periods, not simultaneous random assignment. To evaluate review quality, distinguish whether reviewers accept or reject advice from whether their decisions are correct, using independently assessed cases where possible.

Operate the exception workload

An exception requires handling outside the normal path. An exception queue is tracked work awaiting that handling. A planned wait for an expected document is not automatically an exception; an expired deadline, conflicting information or an unavailable application may require one. Route by what can remedy the problem, not by a single generic error flag.

UiPath's business/application distinction is a useful starting point; ambiguity, policy disposition and uncertain external effects require additional treatment.
ConditionUseful responseReturn or exit condition
Missing or invalid informationRequest correction from the appropriate sourceRequired information becomes usable; unchanged input is not repaired by retrying
Technical application failureInvestigate or retry under the operation's policyDependency recovers and repetition is justified
Ambiguous intentAsk the responsible person to clarifyThe intended result and target are explicit
Known policy rejectionRecord the reason and notify the appropriate partyTerminate normally unless an authorized exception process applies
External effect unconfirmedAssign reconciliation workAuthoritative evidence resolves the effect, or an owner accepts further investigation

Each queue item needs an owner, reason, supporting context, priority, deadline and permitted disposition. Queue age is time since entry; backlog is outstanding work. A worklist without those meanings can hide abandoned cases. UiPath's review worklist illustrates assigned items with distinct review and retry dispositions. A recorded disposition still needs to mean something concrete about the underlying business problem.

Capacity constrains automation

Compare arriving handling effort with the effort reviewers can supply over the same period. Case counts alone are insufficient when difficult cases take much longer. If required work persistently exceeds capacity, the backlog grows even when the automated stages are fast. Amazon's queue-backlog guidance describes prioritization, isolation and backpressure—reducing accepted upstream work when consumers fall behind. Applied to human work, pacing must respect service obligations; urgent or old cases cannot simply be discarded.

Case counts are not handling capacity

Fictional five-day effort accounting: equal case counts can require different handling effort. Daily input is held constant, and unfinished effort carries forward.

Incoming effort: 300 min/day. Available effort: 240 min/day.
Outstanding after five days: 300 min (5 hours).
Outstanding effort, not individual waiting timeInitial outstanding effort followed by five end-of-day values. The line accounts for incoming effort minus handled effort; remaining work is never discarded.Outstanding effort (minutes)0150300InitialDay 1Day 2Day 3Day 4Day 5Days 1–5: end-of-day backlog
DayStart (min)Incoming (min)Handled (min)Remaining (min)
1030024060
260300240120
3120300240180
4180300240240
5240300240300

End-of-day backlog = max(0, previous backlog + incoming minutes − available minutes). Capacity beyond available work goes unused. These totals do not predict individual queue age, deadline attainment or review quality.

Static default calculation

20 cases/day × (75% × 10 + 25% × 30 minutes) = 300 minutes/day. Four reviewer-hours supply 240 minutes/day. Starting with zero, outstanding effort is 60, 120, 180, 240 and 300 minutes after days 1–5.

Case mix and handling time determine incoming effort. Unfinished work remains outstanding; increasing capacity in this arithmetic does not establish review quality or a staffing recommendation.

Universal sign-off can create an unmanageable queue instead of meaningful verification. Reduce avoidable exceptions through better inputs and clearer rules, reserve review for decisions that need it, and keep the actual review burden visible. Paperclip's liveness discussion frames this as a tension between continuing useful work and enforcing valid progression, not a reason to abandon either.

Manual takeover must address execution as well as assignment. Stop new automated commitments, determine which attempts remain in flight and reconcile their effects before conflicting manual work proceeds. Cancellation can be cooperative, and a timed-out attempt may continue. Marking a task human-owned therefore does not establish exclusive control over external records. The implementation must separately prevent conflicting writes at those boundaries.

Integration and recovery

Preserve meaning across applications

An integration contract states what information and operations cross an application boundary and what their results mean. It must cover record identity, field meanings, permitted reads and writes, freshness, operating limits and confirmation. A connector that transports data successfully can still select the wrong account, misinterpret a status or submit an unauthorized change. Preserving meaning across integrations explains the domain agreement; authorization at the protected operation explains enforcement.

Robotic process automation, or RPA, performs prescribed interactions with existing software, often through its graphical interface; it need not use AI. Mary Lacity, Leslie Willcocks and Andrew Craig's April 2015 Telefónica O2 case study describes a 2010 trial handling SIM swaps and precalculated credits by logging into applications and processing transactions. O2 considered RPA alongside process-management software and customer self-service, while retaining some work manually. Interface automation was an implementation option, not a replacement for process design.

Choose the interface from the workflow requirement. For an API, establish supported operations, identity mapping, permissions, limits and result semantics. For notifications, establish delivery and freshness behavior. For batch exchange, define the acceptable delay and per-record acceptance reporting. For graphical automation, identify the intended controls and how their results will be checked. These are requirements to verify for the actual integration, not guarantees implied by the interface category.

When several interfaces retrieve equivalent information, convert their outputs into a shared format before applying business rules. This separates how information is obtained from what decision it supports. In Risa Labs' oncology workflow, an API or automated portal interaction retrieves insurance coverage information. Both paths produce the same coverage-result format for a deterministic check of whether coverage is active; a failed check stops progression. The shared format lets both paths use the same rule, but cannot establish that the retrieved values are correct. Computer Use explains graphical execution and verification.

The architecture described by Risa Labs uses alternative API and portal access paths, then a shared format and active-coverage rule. Normalization makes one rule possible; it does not verify the retrieved values.

Confirmation must reach the business boundary. Under HTTP semantics, 202 Accepted means processing is incomplete, not that the requested change succeeded. Likewise, repairing a broken interface selector restores a way to attempt an action, not proof of its result. Microsoft's UI automation repair guidance describes operator review and testing of proposed selector repairs during debugging. Maintenance and business-result verification remain separate work.

Recover from the effects that occurred

Recovery starts from three different claims: confirmed success, confirmed failure without an effect, or an unknown outcome. A missing response cannot distinguish them. Reconciliation resolves the uncertainty or discrepancy against authoritative records. Preserve the logical operation identifier and determine what happened before selecting the next business action. Recover when the effect is unknown develops the interface mechanics.

Consider an attempted external record creation whose response times out. The record might exist, or the operation might not have committed. An authoritative lookup or operation-status check can justify continuing from an existing record; confirmed non-execution can justify considering another authorized attempt. An unavailable lookup or incomplete search leaves the outcome unknown. Stripe's low-level error guidance illustrates why even some server errors remain indeterminate and why local operation identifiers help correlate later provider evidence.

Idempotency means repeated attempts do not add another effect beyond the first. It needs enforcement by the receiving operation, not merely a logged key. Temporal's activity documentation identifies the effect/reporting gap: an external effect can occur before a worker dies without recording completion. A retry can then repeat it. Recorded completion protects replay; unrecorded completion still requires safe repetition or reconciliation.

Same local symptom, different external states

Same local symptom, different external statesTwo alternative histories of logical create O1 have no success recorded. In A the application commits, returns a result, then the worker dies before recording completion; the record remains. In B the request is lost before arrival and never commits. Both local outcomes are unknown until reconciliation.A · committed, completion unreportedWorkerReceiving applicationExecution historyRequest · create O1Record committedResult · record IDCompletion report lost · worker diesCommitted record remainsNo recorded successSame local observation: O1 outcome unknown.B · request never arrives, no commitWorkerReceiving applicationExecution historyRequest · O1 lostNo record createdAttempt times outNo recorded successSame local observation: O1 outcome unknown.

Reconcile the same O1 against authoritative receiving-system state. A timeout does not stop an in-flight operation, and an incomplete search cannot prove non-execution.

Required effect confirmed

Continue remaining work using the existing result.

Non-execution authoritatively confirmed

Consider a repeat only with valid authority and safe operation semantics.

Unavailable or inconclusive

Retain owned investigation; the outcome stays unknown.

These are alternative histories of the same logical operation O1. Losing a completion report does not undo an external commit. Reconciliation distinguishes confirmed effect, confirmed non-execution and unresolved investigation; recording O1 alone does not enforce idempotency.

Partial completion is still real

Compensation performs a new business-specific action to address completed work. It is not database rollback: concurrent changes must survive, the original state may be unattainable and cancellation can incur charges. Record completed effects and the information needed to compensate them. Compensation can itself fail, so preserve its progress and assign manual recovery when necessary. For irreversible effects, define an acceptable remedy rather than claiming erasure. Microsoft's Compensating Transaction pattern explains these boundaries.

If one application has committed and another has failed, recover those specific obligations rather than restarting the whole business process. A completed charge followed by failed receipt delivery calls for recovering the receipt step, not casually charging again. Eric Allam's replay example illustrates preserving completed steps, subject to the external-effect gap just described.

Durable execution preserves logical progress across interruptions, including long human waits. It can let a process resume without keeping one worker alive. It does not establish that external effects were reversed when execution was cancelled, or that an old approval is still valid. On resumption, recheck relevant business state and authorization; investigate late completions rather than discarding them. Persistence, replay and retry implementation belong in Agent Runtimes and Harness Engineering.

Outcomes and daily operation

Measure the complete process

Return to the case boundary and customer baseline. Measurement should describe useful completed work, including checking, waiting and correction. Klarity's outcome-first evaluation approach works backward from user value to utilization and feature-health checks; a valid output format is one useful check, not a substitute for the user's result.

Record these dimensions separately for the baseline and redesigned process. Use the same eligibility rules, work mix and observation window; leave unavailable observations explicit.
MeasureDefinition for the comparisonWhat it can hide alone
Elapsed completion timeTime from the defined start to accepted completion, including waitingStalled cases absent from a completed-only sample
Active handling effortPerson-time spent producing, checking, correcting and recovering workConcurrent effort if mistaken for elapsed time
ThroughputAccepted useful completions per unit of timeA changing case mix or weakened acceptance standard
First-pass yieldCases meeting requirements without repair divided by cases entering the defined processEventual quality after correction; unresolved cases need separate visibility
Straight-through coverageCases completed without human intervention divided by eligible entering casesWhether those results were correct
Unresolved workOutstanding cases, their ages, reasons and owners at the observation cutoffNothing should silently convert these cases into successes
Operating cost per accepted completionAttributed operating costs, including review, repair and integration maintenance, divided by accepted completionsUnfinished work and one-time delivery costs unless reported alongside

ASQ's first-pass-yield definition excludes work requiring reruns, retesting or repair. A case can therefore complete eventually without passing on its first attempt. Likewise, a fully automatic path can be wrong, while a human-assisted path can meet the requirement immediately. If two people work simultaneously for ten minutes, the process consumes twenty person-minutes but only ten elapsed minutes during that interval. Keep these quantities separate.

Interpret the comparison

Generative AI at Work, by Erik Brynjolfsson, Danielle Li and Lindsey Raymond, supplies a bounded example of useful assistance. Its November 2024 revision studies staggered deployment among 5,172 support workers and reports a 15% average increase in issues resolved per hour. Workers could edit or ignore suggested responses. Effects differed: less experienced and lower-skilled workers improved speed and quality, while the most experienced and highest-skilled workers had small speed gains and small quality declines. This is evidence about one firm's assisted workflow, not autonomous fulfillment or net operating savings.

Released capacity is not automatically reduced expenditure. If staff remain employed and use freed time for more work, capacity has increased without a payroll reduction. The Government Efficiency Framework distinguishes these benefits and requires realized savings to account for delivery costs and costs shifted elsewhere. Report the actual benefit: shorter waits, more completions, better quality, less effort or lower spending.

A credible comparison must also account for changes in case difficulty, staffing and policy. Faster completed cases do not establish improvement if hard cases accumulate unresolved. Choose an appropriate live experiment, and connect execution records to actual outcomes through Observability. Component speed is useful for diagnosis; the process boundary determines the benefit claim.

Keep the workflow usable and owned

Adoption means sustained incorporation into eligible work. An operational measure is the share of eligible cases actually handled through the intended arrangement over a stated period, accompanied by bypasses and duplicate manual work. Account creation and occasional use answer different questions. Normalization Process Theory emphasizes the everyday interactions, confidence, accountability, division of labor and resources needed to make a practice workable.

Observation-only operation, assistance and narrowly authorized automation are legitimate long-term choices. Shadow mode compares a candidate's behavior without adopting its outputs. SageMaker shadow testing, for example, copies inference requests to a candidate while returning production responses to the application. That response-routing contract does not make a shadow agent's tool calls harmless. During comparison, keep one authorized path for real effects and prevent the candidate from duplicating them.

Operators need practical training, a usable manual route and support that can resolve problems. If they routinely repeat the entire task to trust the output, include that effort in the result and investigate its cause. Preserving a recognizable process can help people understand their new responsibilities, but familiarity should not protect unnecessary work. Test the arrangement with its actual operators.

Assign responsibility for maintaining the process, not merely for launching its software.
Observed problemResponsible rolePossible change
Cases bypass the intended pathProcess owner with operatorsRevise eligibility, task fit or unnecessary handoffs
Rules no longer match the workDomain decision ownerReview policy and update its executable checks
Application changes break accessIntegration ownerRepair and verify the affected boundary
Exceptions age without actionOperations ownerAdjust staffing, priorities, escalation or admitted scope
Operators repeatedly correct the same outputQuality owner with domain specialistsIdentify whether the source, rule, interface or model behavior needs repair

Corrections need an explicit destination. Fixing the present artifact, reporting a defect and changing future model behavior are different operations. Google's Feedback + Control guidance recommends promising only the timing and scope of improvement the system can deliver. Human feedback does not automatically train a model. A recurring problem may instead require a source correction, clearer rule, better interface or separately evaluated model change.

Use operating handoff to establish that the receiving team can run and recover the process; use Enterprise AI for wider organizational adoption. Then revisit the original allocation using observed completion, quality, burden and maintenance. Retain automation where it helps, revise the arrangement where it moves the bottleneck, and stop it where the complete process is worse. Greater autonomy is one possible change, not the destination.

Open questions

  1. Sustained net benefit remains difficult to establish when task difficulty, review effort, staffing and adoption change together. Progress would mean longitudinal comparisons that retain unresolved cases and measure accepted completion, correction burden and full operating cost on a consistent eligible population.

  2. Manual takeover across multiple applications remains harder than assigning a human task. In-flight automation and independent writers can outlive that assignment. Progress would mean implementation-specific demonstrations that prevent conflicting commitments while reconciling attempts whose effects are still unknown.

  3. Review allocation must preserve error detection without producing a queue people cannot meaningfully inspect. The difficulty is that review policy changes both the selected workload and reviewer behavior. Progress would combine error-detection tests with queue age, handling effort and downstream outcomes, rather than optimizing approval rate alone.

  4. Process records reveal only the work that systems capture. Informal clarification, duplicate checking and off-system repair can make a redesigned path appear better than it is. Progress would combine event-based analysis with targeted observation and explicit accounting for omitted work.

Follow the curated reading path through the speakers and demonstrations behind this entry.

15 min

AI Engineer Summit 2025 · 2025

Missing pieces of workflow automation

Shirsha Chaudhuri

Cited in this entry

Use its support and content examples to recognize when task-by-task automation leaves the original handoffs and approval structure untouched. It motivates redesign with domain specialists.

Watch talk
18 min

AI Engineer World's Fair 2026 · 2026

Your Agent Didn’t Fail. Your Harness Did.

Vinoth Govindarajan

Cited in this entry

Connects workflow responsibility to state ownership, scoped approval and records of actual external effects. Useful when investigating a process that appears successful but cannot establish what happened.

Watch talk
28 min

AI Engineer World's Fair 2024 · 2024

E-Values: Evaluating the Values of AI

Sheila Gulati · Nischal Nadhamuni

Cited in this entry

Shows how to work backward from user outcomes to an evaluation collection, instead of treating convenient feature-health metrics as business value.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

72 matching talks

TalkSpeakerEventYear
Balázs HorváthAI Engineer World's Fair 20262026
Cornelia DavisAI Engineer World's Fair 20262026
Martin Harrysson, Natasha ManiarAI Engineer Code 20252025
Eliza Cabrera, Jeremy SilvaAI Engineer World's Fair 20252025
DottaAI Engineer World's Fair 20262026
Dan MasonAI Engineer World's Fair 20252025
Rachna SrivastavaAI Engineer World's Fair 20252025
Dan BjornnAI Engineer World's Fair 20262026
Steven WillmottAI Engineer Europe 20262026
Elizabeth Fuentes LeoneAI Engineer World's Fair 20262026
Anju KambadurAI Engineer Summit 20252025
Dan FarrellyAI Engineer World's Fair 20262026
Ben SteinAI Engineer World's Fair 20252025
Angel Ortmann LeeAI Engineer World's Fair 20262026
Jeremy Silva, Chris HernandezAI Engineer World's Fair 20252025
Keegan McCallumAI Engineer World's Fair 20252025
Anant ShankhdharAI Engineer World's Fair 20262026
Sandipan BhaumikAI Engineer Europe 20262026
Eric AllamAI Engineer World's Fair 20252025
Justin ReockAI Engineer Code 20252025
Sachin GuptaAI Engineer World's Fair 20262026
Nikhil AbrahamAI Engineer World's Fair 20252025
Mike SpitzAI Engineer Europe 20262026
Anna Marie BenzonAI Engineer World's Fair 20262026
Alex VolkovAI Engineer World's Fair 20262026
Vasuman MozaAI Engineer World's Fair 20262026
Garrett GalowAI Engineer Europe 20262026
Gabe De MesaAI Engineer World's Fair 20262026
Luke AlvoeiroAI Engineer Europe 20262026
Notion's Token Town

Transcript reviewed

Sarah SachsAI Engineer World's Fair 20262026
Kyle MisteleAI Engineer World's Fair 20262026
Mason EggerAI Engineer World's Fair 20252025
Jia WuAI Engineer World's Fair 20262026
How to Kill the Code Review

Transcript reviewed

Ankit JainAI Engineer World's Fair 20262026
Maggie AppletonAI Engineer Europe 20262026
Content Is Code

Transcript reviewed

Matt PalmerAI Engineer World's Fair 20262026
Chaitanya AsawaAI Engineer World's Fair 20262026
Sarthak AggarwalAI Engineer World's Fair 20262026
Samuel ColvinAI Engineer Code 20252025
Cornelia DavisAI Engineer Code 20252025
Yohei NakajimaAI Engineer World's Fair 20262026
Apoorva JoshiAI Engineer World's Fair 20262026
Sarmad QadriAI Engineer World's Fair 20252025
Nicholas ArcolanoAI Engineer Code 20252025
Dan FengAI Engineer World's Fair 20262026
Melanie WarrickAI Engineer World's Fair 20262026
Evan KormosEvent unknownUnknown
Jay Sahnan, Arik BirdEvent unknownUnknown
Felippe Felisola Caso, Francesco GallettaEvent unknownUnknown
Dmitry BuykinAI Engineer World's Fair 20262026
Sait IzmitAI Engineer World's Fair 20262026
Hari KumarEvent unknownUnknown
Dan Barrett, Raymond Weitekamp, Jose Montes de OcaEvent unknownUnknown
Howie Liu, Shawn "swyx" Wang, Joshua XuAI Engineer World's Fair 20262026
Vincent WendyEvent unknownUnknown
Flora LiuAI Engineer World's Fair 20262026
Michael Manapat, Jack McDonaldEvent unknownUnknown
Prathik NaiduEvent unknownUnknown
Taylor Luttrell-Williams, Killian Carlsen-PhelanEvent unknownUnknown
Zack Proser, Nick NisiEvent unknownUnknown
Shy RuparelAI Engineer New York 20262026
Charlie Guo, Gabriel ChuaAI Engineer World's Fair 20262026
Jack WotherspoonEvent unknownUnknown
Rémi LoufAI Engineer World's Fair 20262026
Priyanka Phatak, Gabe CemajAI Engineer World's Fair 20262026
Karthic RaoAI Engineer NYC 2026 Online Talks2026
Viren BaraiyaAI Engineer World's Fair 20262026
Shubhankar SrivastavaAI Engineer World's Fair 20262026
Everett BerryAI Engineer World's Fair 20262026
Gabe Cemaj, Priyanka PhatakAI Engineer World's Fair 20262026
Karan VaidyaEvent unknownUnknown
Aman SinghEvent unknownUnknown

References

Coverage and source review
Processed transcripts
50 processed in full · 5 in the curated path
Automated source review
Passed
Metadata candidates
27 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. ISO 9000: Quality Management Vocabulary

    https://www.iso.org/obp/ui?_escaped_fragment_=iso:std:iso:9000:ed-5:v1:en

  2. Missing pieces of workflow automation

    https://www.youtube.com/watch?v=xq9sz5MksVI&t=302

  3. OMG: Case Management Model and Notation, Version 1.1

    https://www.omg.org/spec/CMMN/1.1/PDF

  4. Zendesk: What is the difference between a solved ticket and a closed ticket?

    https://support.zendesk.com/hc/en-us/articles/4408887712154-What-is-the-difference-between-a-solved-ticket-and-a-closed-ticket

  5. Missing pieces of workflow automation

    https://www.youtube.com/watch?v=xq9sz5MksVI&t=619

  6. Toyota Production System

    https://global.toyota/en/company/vision-and-philosophy/production-system/

  7. The New Industrial Engineering: Information Technology and Business Process Redesign

    https://www.scribd.com/document/797780246/The-New-Industrial-Engineering-Davenport

  8. Workflow Management Coalition: The Workflow Reference Model

    https://www.aiai.ed.ac.uk/project/wfmc/ARCHIVE/DOCS/refmodel/rmv1-16.html

  9. A Model for Types and Levels of Human Interaction with Automation

    https://www.researchgate.net/publication/11596569_A_model_for_types_and_levels_of_human_interaction_with_automation

  10. Lean Enterprise Institute: Value Stream Mapping

    https://www.lean.org/lexicon-terms/value-stream-mapping/

  11. AHRQ: Mapping and Redesigning Workflow

    https://www.ahrq.gov/sites/default/files/wysiwyg/ncepcr/tools/PCMH/pcpf-module-10-workflow-mapping.pdf

  12. Leadership in AI-Assisted Engineering

    https://www.youtube.com/watch?v=PmZDupFP3UM&t=899

  13. One Developer, Two Dozen Agents, Zero Alignment: Why we Need Collaborative AI Engineering

    https://www.youtube.com/watch?v=ClWD8OEYgp8&t=264

  14. Process Mining Manifesto

    https://research.wu.ac.at/ws/portalfiles/portal/19821300/vanderAalst_etal_2011_BPM_Process-Mining-Manifesto.pdf

  15. Building Applications with AI Agents

    https://www.youtube.com/watch?v=R30col3UPUg&t=495

  16. AI tools for Forward Deployed Engineering

    https://www.youtube.com/watch?v=l0FLhNqBOic&t=384

  17. Building Applications with AI Agents

    https://www.youtube.com/watch?v=R30col3UPUg&t=405

  18. Robotic Process Automation at Telefónica O2

    https://researchonline.lse.ac.uk/id/eprint/64516/1/OUWRPS_15_02_published.pdf

  19. The Build-Operate Divide: Bridging Product Vision and AI Operational Reality

    https://www.youtube.com/watch?v=1__V4KTv_Gw&t=251

  20. Decision Model and Notation, Version 1.3

    https://www.omg.org/spec/DMN/1.3/PDF

  21. Anthropic: Building effective agents

    https://www.anthropic.com/engineering/building-effective-agents

  22. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    https://www.youtube.com/watch?v=7P0elyLIxXo&t=219

  23. IT Admin for the AI Workforce — Sarthak Aggarwal, Decawork

    https://www.youtube.com/watch?v=q-WOjZhOMCA&t=676

  24. Challenges to Scaling Agents for Generative AI Products

    https://www.youtube.com/watch?v=b2GqTDWtg6s&t=550

  25. IBM Redbooks: Building the Operational Data Store on DB2 UDB

    https://www.redbooks.ibm.com/redbooks/pdfs/sg246513.pdf

  26. Your Agent Didn’t Fail. Your Harness Did.

    https://www.youtube.com/watch?v=BInpv7lGp1o&t=159

  27. Your Agent Didn’t Fail. Your Harness Did.

    https://www.youtube.com/watch?v=BInpv7lGp1o&t=58

  28. Your Agent Didn’t Fail. Your Harness Did.

    https://www.youtube.com/watch?v=BInpv7lGp1o&t=882

  29. From Stateless Nightmares to Durable Agents

    https://www.youtube.com/watch?v=flf_IKnFYnE&t=532

  30. RFC 9110: HTTP Semantics

    https://www.rfc-editor.org/rfc/rfc9110.html

  31. Your Agent Didn’t Fail. Your Harness Did.

    https://www.youtube.com/watch?v=BInpv7lGp1o&t=508

  32. Exposing Agents as MCP Servers with mcp-agent: Sarmad Qadri

    https://www.youtube.com/watch?v=uFPAtKIN-FQ&t=574

  33. W3C: State Chart XML—State Machine Notation for Control Abstraction

    https://www.w3.org/TR/scxml/

  34. Stripe: Receive events in your webhook endpoint

    https://docs.stripe.com/webhooks

  35. Web Services—Human Task Specification Version 1.1, Committee Specification 01

    https://docs.oasis-open.org/bpel4people/ws-humantask-1.1-spec-cs-01.html

  36. PagerDuty: Incidents

    https://support.pagerduty.com/main/docs/incidents

  37. Business Process Model and Notation, Version 2.0.2

    https://www.omg.org/spec/BPMN/2.0.2/PDF

  38. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    https://www.youtube.com/watch?v=7P0elyLIxXo&t=263

  39. AI System Design: From Idea to Production

    https://www.youtube.com/watch?v=T0HhO4YtTfE&t=1122

  40. The Build-Operate Divide: Bridging Product Vision and AI Operational Reality

    https://www.youtube.com/watch?v=1__V4KTv_Gw&t=345

  41. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    https://www.youtube.com/watch?v=tDArkCqjA-c&t=3045

  42. OWASP Transaction Authorization Cheat Sheet

    https://cheatsheetseries.owasp.org/cheatsheets/Transaction_Authorization_Cheat_Sheet.html

  43. When Machines Mislead: Human Review of Erroneous AI Cheating Signals

    https://aclanthology.org/2025.aimecon-wip.11.pdf

  44. UiPath Orchestrator: Business Exception Vs Application Exception

    https://docs.uipath.com/orchestrator/automation-cloud/latest/user-guide/business-exception-vs-application-exception

  45. AI tools for Forward Deployed Engineering

    https://www.youtube.com/watch?v=l0FLhNqBOic&t=225

  46. Temporal Activity Execution

    https://docs.temporal.io/activity-execution

  47. UiPath Orchestrator: Review Requests

    https://docs.uipath.com/orchestrator/automation-cloud/latest/user-guide/managing-review-requests

  48. Amazon Builders’ Library: Avoiding insurmountable queue backlogs

    https://d1.awsstatic.com/builderslibrary/pdfs/avoiding-insurmountable-queue-backlogs.pdf

  49. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    https://www.youtube.com/watch?v=7P0elyLIxXo&t=120

  50. Your Agent Didn’t Fail. Your Harness Did.

    https://www.youtube.com/watch?v=BInpv7lGp1o&t=717

  51. Microsoft Power Automate: AI-assisted UI automation repair

    https://learn.microsoft.com/en-us/power-automate/desktop-flows/ai-assisted-ui-automation-repair

  52. Can Oncology Workflows Run Without Human Touch? - Anant Shankhdhar, Risa Labs

    https://www.youtube.com/watch?v=_cVfz88_j7A&t=232

  53. Resolving an ambiguous payment request

    https://docs.stripe.com/error-low-level

  54. Temporal Activity Definition

    https://docs.temporal.io/activity-definition

  55. Compensating Transaction pattern

    https://learn.microsoft.com/en-us/azure/architecture/patterns/compensating-transaction

  56. Two Roads to Durable Agents: Replay vs. Snapshot — Eric Allam, Co-founder, Trigger.dev

    https://www.youtube.com/watch?v=svCnShDvgQg&t=207

  57. Building Durable, Production-Ready Agents with OpenAI SDK and Temporal

    https://www.youtube.com/watch?v=k8cnVCMYmNc&t=3485

  58. E-Values: Evaluating the Values of AI

    https://www.youtube.com/watch?v=mpzktdYTcdE&t=1053

  59. Brynjolfsson, Li, and Raymond: Generative AI at Work

    https://arxiv.org/pdf/2304.11771

  60. ASQ: Quality Glossary—First pass yield

    https://asq.org/quality-resources/quality-glossary

  61. The Government Efficiency Framework

    https://www.gov.uk/government/publications/the-government-efficiency-framework/the-government-efficiency-framework--2

  62. What Data from 20 Million Pull Requests Reveal About AI Transformation

    https://www.youtube.com/watch?v=WqZq8L-v9pA&t=388

  63. What Data from 20 Million Pull Requests Reveal About AI Transformation

    https://www.youtube.com/watch?v=WqZq8L-v9pA&t=414

  64. Normalization Process Theory: Collective Action

    https://normalization-process-theory.northumbria.ac.uk/what-is-npt/collective-action/

  65. Amazon SageMaker AI: Shadow tests

    https://docs.aws.amazon.com/sagemaker/latest/dg/shadow-tests.html

  66. The Build-Operate Divide: Bridging Product Vision and AI Operational Reality

    https://www.youtube.com/watch?v=1__V4KTv_Gw&t=129

  67. People + AI Guidebook: Feedback + Control

    https://pair.withgoogle.com/guidebook-v2/chapter/feedback-controls/

  68. Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCP

    https://www.youtube.com/watch?v=sn79oS4MZFI&t=4076

  69. NIST AI RMF Core

    https://airc.nist.gov/airmf-resources/airmf/5-sec-core/