[
  {
    "startMs": 800,
    "text": "Hey folks, this is Gish, a principal"
  },
  {
    "startMs": 3840,
    "text": "software engineer at Microsoft. Outside"
  },
  {
    "startMs": 6000,
    "text": "my job, I build open source tools for AI"
  },
  {
    "startMs": 9120,
    "text": "agents. One of them is MCP pacemaker, a"
  },
  {
    "startMs": 12559,
    "text": "supervised bridge that keeps MCP servers"
  },
  {
    "startMs": 15120,
    "text": "running."
  },
  {
    "startMs": 16880,
    "text": "Before we start, a small disclosure. The"
  },
  {
    "startMs": 19520,
    "text": "views in this talk are my own and"
  },
  {
    "startMs": 21600,
    "text": "doesn't represent my employer."
  },
  {
    "startMs": 24160,
    "text": "So, sub agents are a distributed system."
  },
  {
    "startMs": 27279,
    "text": "Over the past year, I have been using"
  },
  {
    "startMs": 29279,
    "text": "agents and sub agents on real"
  },
  {
    "startMs": 30880,
    "text": "engineering work, dev sessions, code"
  },
  {
    "startMs": 33840,
    "text": "reviews, instant trashing,"
  },
  {
    "startMs": 36480,
    "text": "whatnot."
  },
  {
    "startMs": 38480,
    "text": "This talk is about what happens when one"
  },
  {
    "startMs": 41200,
    "text": "agent hands work to another."
  },
  {
    "startMs": 44640,
    "text": "So, here's an architecture."
  },
  {
    "startMs": 47440,
    "text": "The orchestrator splits a task and hands"
  },
  {
    "startMs": 50239,
    "text": "the pieces to three different sub aents."
  },
  {
    "startMs": 53760,
    "text": "They all run at the same time."
  },
  {
    "startMs": 56559,
    "text": "They all read from the same shared"
  },
  {
    "startMs": 58719,
    "text": "state. Consization, memory files, repo,"
  },
  {
    "startMs": 63120,
    "text": "working trees, issues, vector stores."
  },
  {
    "startMs": 68000,
    "text": "And they don't just think, they make"
  },
  {
    "startMs": 70560,
    "text": "tool calls to real systems"
  },
  {
    "startMs": 74560,
    "text": "which touch databases, APIs, CI tools,"
  },
  {
    "startMs": 78080,
    "text": "deployment pipelines"
  },
  {
    "startMs": 80479,
    "text": "and"
  },
  {
    "startMs": 82159,
    "text": "the results come back and orchestrator"
  },
  {
    "startMs": 85119,
    "text": "merges them into a single answer. An"
  },
  {
    "startMs": 87439,
    "text": "orchestrator workers shared mutable"
  },
  {
    "startMs": 90479,
    "text": "state and results merge from concurrent"
  },
  {
    "startMs": 93200,
    "text": "executions. This represents a classic"
  },
  {
    "startMs": 95600,
    "text": "distributed system."
  },
  {
    "startMs": 97680,
    "text": "I have known how to reason with dispute"
  },
  {
    "startMs": 99600,
    "text": "systems for decades,"
  },
  {
    "startMs": 101840,
    "text": "but none of the setup came with it."
  },
  {
    "startMs": 105759,
    "text": "What you get is a function that takes a"
  },
  {
    "startMs": 108159,
    "text": "string and returns a string."
  },
  {
    "startMs": 110960,
    "text": "Failures we hit are not new, are not"
  },
  {
    "startMs": 114000,
    "text": "specific to a agents. They're classic"
  },
  {
    "startMs": 116640,
    "text": "distributed system failures."
  },
  {
    "startMs": 120159,
    "text": "partial failure, a normal call that"
  },
  {
    "startMs": 122880,
    "text": "fails halfway through and returns an"
  },
  {
    "startMs": 125680,
    "text": "exception error code. In a sub agent"
  },
  {
    "startMs": 128640,
    "text": "case, it returns text that looks like a"
  },
  {
    "startMs": 131440,
    "text": "success."
  },
  {
    "startMs": 133120,
    "text": "Dirty reads normally we have isolations"
  },
  {
    "startMs": 136879,
    "text": "versions. Here every worker reads one"
  },
  {
    "startMs": 140000,
    "text": "shade context including things other"
  },
  {
    "startMs": 142560,
    "text": "worker only guessed."
  },
  {
    "startMs": 145040,
    "text": "Unbounded fan out."
  },
  {
    "startMs": 147520,
    "text": "A busy service pushes [clears throat]"
  },
  {
    "startMs": 150000,
    "text": "back with the cues 429's 503s."
  },
  {
    "startMs": 154800,
    "text": "In agents, orchestrator does not see any"
  },
  {
    "startMs": 157920,
    "text": "of this. It just starts in the worker."
  },
  {
    "startMs": 161040,
    "text": "Retries."
  },
  {
    "startMs": 162959,
    "text": "In normal distute systems, we build"
  },
  {
    "startMs": 165599,
    "text": "potency to make retries reliable."
  },
  {
    "startMs": 168800,
    "text": "in agents a reiterate sub agent runs the"
  },
  {
    "startMs": 171920,
    "text": "whole task again with side effects"
  },
  {
    "startMs": 174000,
    "text": "included"
  },
  {
    "startMs": 175519,
    "text": "for each of these case"
  },
  {
    "startMs": 178400,
    "text": "I'll name the concept it already is then"
  },
  {
    "startMs": 181360,
    "text": "I'll put them together as a"
  },
  {
    "startMs": 182747,
    "text": "[clears throat] consistency model I'm"
  },
  {
    "startMs": 184000,
    "text": "proposing and walk through why each part"
  },
  {
    "startMs": 186640,
    "text": "is there"
  },
  {
    "startMs": 188560,
    "text": "start with the failure model because"
  },
  {
    "startMs": 190480,
    "text": "everything else depends on it a friendly"
  },
  {
    "startMs": 193440,
    "text": "case is a crash stop a process it was"
  },
  {
    "startMs": 196319,
    "text": "running correctly stops and stays is"
  },
  {
    "startMs": 198480,
    "text": "stopped. The subway agent that throws an"
  },
  {
    "startMs": 200959,
    "text": "error in this box. It's easy to handle."
  },
  {
    "startMs": 204560,
    "text": "Next, no reply. The call is lost or it's"
  },
  {
    "startMs": 207840,
    "text": "very late. We'll come back to this soon."
  },
  {
    "startMs": 211840,
    "text": "The hot case is the one from Baantine"
  },
  {
    "startMs": 215280,
    "text": "general sniper. A component that gives"
  },
  {
    "startMs": 218480,
    "text": "conflicting information. The process"
  },
  {
    "startMs": 220720,
    "text": "answers and the answer is wrong."
  },
  {
    "startMs": 223760,
    "text": "A half finished sub agent sits closest"
  },
  {
    "startMs": 225840,
    "text": "to the box. It is not malicious but from"
  },
  {
    "startMs": 228640,
    "text": "the outside it looks the same. It does"
  },
  {
    "startMs": 230959,
    "text": "not crash. It answers fluently. Let's"
  },
  {
    "startMs": 233519,
    "text": "jump into this. A sub agent that half"
  },
  {
    "startMs": 237200,
    "text": "finished does not raise. It returns"
  },
  {
    "startMs": 240000,
    "text": "normal looking text with the same type"
  },
  {
    "startMs": 242400,
    "text": "signature as success."
  },
  {
    "startMs": 244560,
    "text": "I talked about one version of this in"
  },
  {
    "startMs": 247280,
    "text": "one of my talks. We pointed an agent at"
  },
  {
    "startMs": 250319,
    "text": "an LQ."
  },
  {
    "startMs": 252159,
    "text": "Here's an illustration I used there with"
  },
  {
    "startMs": 254720,
    "text": "three different paths which gave the"
  },
  {
    "startMs": 256639,
    "text": "same answer."
  },
  {
    "startMs": 258320,
    "text": "The first one"
  },
  {
    "startMs": 260320,
    "text": "it pulled the telemetry and compared"
  },
  {
    "startMs": 262400,
    "text": "with a nonbrand set. It actually did the"
  },
  {
    "startMs": 265120,
    "text": "work. The second one it matched against"
  },
  {
    "startMs": 268160,
    "text": "the title. It it has seen before and"
  },
  {
    "startMs": 271600,
    "text": "gave the response."
  },
  {
    "startMs": 273840,
    "text": "The third alert"
  },
  {
    "startMs": 275919,
    "text": "it didn't do anything."
  },
  {
    "startMs": 278160,
    "text": "It just hallated."
  },
  {
    "startMs": 281280,
    "text": "All three are categorized as benign with"
  },
  {
    "startMs": 283360,
    "text": "the same level of confidence"
  },
  {
    "startMs": 285840,
    "text": "and nothing downstream can tell these"
  },
  {
    "startMs": 287919,
    "text": "apart. The orchestrator can read the"
  },
  {
    "startMs": 290240,
    "text": "conclusion. It cannot read how much work"
  },
  {
    "startMs": 292400,
    "text": "was behind it."
  },
  {
    "startMs": 294880,
    "text": "Now the call gets no reply."
  },
  {
    "startMs": 298240,
    "text": "The orchestrator send work and the"
  },
  {
    "startMs": 300880,
    "text": "timeout fired. From here three volts"
  },
  {
    "startMs": 304400,
    "text": "look the same. The sub agent is still"
  },
  {
    "startMs": 306800,
    "text": "working. It crashed and the reply was"
  },
  {
    "startMs": 309600,
    "text": "lost."
  },
  {
    "startMs": 311039,
    "text": "It's a hall limit, not a tooling gap."
  },
  {
    "startMs": 313600,
    "text": "The FLP paper from 1985 showed that in a"
  },
  {
    "startMs": 316639,
    "text": "synchronous system with even one faulty"
  },
  {
    "startMs": 319039,
    "text": "process, agreement can fail to finish."
  },
  {
    "startMs": 322560,
    "text": "Chundra and 2x answer in 1996 was"
  },
  {
    "startMs": 325360,
    "text": "failure detectors that can make"
  },
  {
    "startMs": 326960,
    "text": "mistakes. They suspect did not know. I"
  },
  {
    "startMs": 330639,
    "text": "hit both directions"
  },
  {
    "startMs": 333120,
    "text": "in my own open source project MCP"
  },
  {
    "startMs": 336560,
    "text": "pacemaker supervised bridge"
  },
  {
    "startMs": 339520,
    "text": "a server whose first run downloads"
  },
  {
    "startMs": 342080,
    "text": "itself hit the 30 secondond call timeout"
  },
  {
    "startMs": 345360,
    "text": "and look broken it was slow not dead we"
  },
  {
    "startMs": 349120,
    "text": "fixed it by giving initialize its own"
  },
  {
    "startMs": 351759,
    "text": "timeout now the other way the server"
  },
  {
    "startMs": 354880,
    "text": "crashed during a call the call then"
  },
  {
    "startMs": 357600,
    "text": "timed out and the timeout overrode the"
  },
  {
    "startMs": 360560,
    "text": "exit code that explained the crash. We"
  },
  {
    "startMs": 363440,
    "text": "fixed it by recording the time out only"
  },
  {
    "startMs": 366639,
    "text": "when it was alive."
  },
  {
    "startMs": 368800,
    "text": "So"
  },
  {
    "startMs": 370319,
    "text": "time out is a suspicion. Record it as"
  },
  {
    "startMs": 372720,
    "text": "unknown and keep the real cause."
  },
  {
    "startMs": 376319,
    "text": "And now make failure a value."
  },
  {
    "startMs": 381520,
    "text": "Let's take this illustration."
  },
  {
    "startMs": 384000,
    "text": "A sub agent returns a status. what it"
  },
  {
    "startMs": 387280,
    "text": "did, what it did not do, the evidence,"
  },
  {
    "startMs": 390800,
    "text": "and any side effects. Partial is a"
  },
  {
    "startMs": 393280,
    "text": "normal answer here."
  },
  {
    "startMs": 395600,
    "text": "The protocols already have a slot for"
  },
  {
    "startMs": 398080,
    "text": "this. In A2A, a task can end as"
  },
  {
    "startMs": 401759,
    "text": "completed, failed, cancelled, or"
  },
  {
    "startMs": 404160,
    "text": "rejected or stop and ask for input. In"
  },
  {
    "startMs": 407919,
    "text": "MCP, a dual result can say is error"
  },
  {
    "startMs": 411680,
    "text": "true. But a sub agent can still say"
  },
  {
    "startMs": 415199,
    "text": "completed while its text says something"
  },
  {
    "startMs": 417919,
    "text": "else. And a loop that only reads the"
  },
  {
    "startMs": 420560,
    "text": "text never sees the difference."
  },
  {
    "startMs": 423280,
    "text": "Another split."
  },
  {
    "startMs": 425280,
    "text": "Checking a condition and getting no is a"
  },
  {
    "startMs": 428400,
    "text": "different fact from not being able to"
  },
  {
    "startMs": 430560,
    "text": "check at all."
  },
  {
    "startMs": 432639,
    "text": "My own individual internet draft on"
  },
  {
    "startMs": 435280,
    "text": "evaluation states"
  },
  {
    "startMs": 437759,
    "text": "talks about this."
  },
  {
    "startMs": 440000,
    "text": "Collapse the two and you change what"
  },
  {
    "startMs": 442319,
    "text": "gets retrieded and who gets ped and what"
  },
  {
    "startMs": 445199,
    "text": "the audit says later."
  },
  {
    "startMs": 448000,
    "text": "Next dirty reads."
  },
  {
    "startMs": 451360,
    "text": "Let's take this madeup example from one"
  },
  {
    "startMs": 453440,
    "text": "of my talk."
  },
  {
    "startMs": 455440,
    "text": "The sub agent the trash it writes into a"
  },
  {
    "startMs": 459199,
    "text": "shade context"
  },
  {
    "startMs": 460960,
    "text": "saying that error rate fell by 80%. It"
  },
  {
    "startMs": 464080,
    "text": "did not measure that it inferred it."
  },
  {
    "startMs": 467520,
    "text": "Now"
  },
  {
    "startMs": 470160,
    "text": "two other sub aents read from the same"
  },
  {
    "startMs": 472240,
    "text": "shade context."
  },
  {
    "startMs": 474800,
    "text": "The second sub aent writes a review note"
  },
  {
    "startMs": 477599,
    "text": "and picks the same sentence up. The"
  },
  {
    "startMs": 480160,
    "text": "third agent writes a runbook step and"
  },
  {
    "startMs": 482960,
    "text": "uses it as a reason to proceed."
  },
  {
    "startMs": 485919,
    "text": "The orchestrator merges all the all"
  },
  {
    "startMs": 488479,
    "text": "these three three artifacts agree. It is"
  },
  {
    "startMs": 491919,
    "text": "once unsupported sentence read three"
  },
  {
    "startMs": 494400,
    "text": "different times."
  },
  {
    "startMs": 496400,
    "text": "The 1995 critique of SQL isolation"
  },
  {
    "startMs": 498879,
    "text": "levels define a dirty read as you read a"
  },
  {
    "startMs": 501759,
    "text": "value before the writer commits or rolls"
  },
  {
    "startMs": 503759,
    "text": "back. In a shared context, a guess is"
  },
  {
    "startMs": 507120,
    "text": "readable the moment it is written and"
  },
  {
    "startMs": 509280,
    "text": "nothing marks it as unchecked."
  },
  {
    "startMs": 512320,
    "text": "A regular fix"
  },
  {
    "startMs": 514800,
    "text": "to is to ask three sub agents and get a"
  },
  {
    "startMs": 517919,
    "text": "quorum."
  },
  {
    "startMs": 519919,
    "text": "Corums do work for a specific reason. In"
  },
  {
    "startMs": 523120,
    "text": "weighted voting from 1979, the read"
  },
  {
    "startMs": 526160,
    "text": "corum plus right corum is greater than"
  },
  {
    "startMs": 528800,
    "text": "total."
  },
  {
    "startMs": 530320,
    "text": "So every read overlaps the latest right."
  },
  {
    "startMs": 532959,
    "text": "The overlap is the guarantee."
  },
  {
    "startMs": 535519,
    "text": "In sub aent's case, the three sub aents"
  },
  {
    "startMs": 538399,
    "text": "that read the same shared context does"
  },
  {
    "startMs": 540560,
    "text": "not give you that. They are not"
  },
  {
    "startMs": 542320,
    "text": "independent copies. They read the same"
  },
  {
    "startMs": 544560,
    "text": "guess. Their agreement is one source"
  },
  {
    "startMs": 547519,
    "text": "counted three different times. Let's"
  },
  {
    "startMs": 549760,
    "text": "pick a consistency model."
  },
  {
    "startMs": 552240,
    "text": "In databases, we have vocabulary for"
  },
  {
    "startMs": 554399,
    "text": "this."
  },
  {
    "startMs": 556000,
    "text": "Linearizable."
  },
  {
    "startMs": 557760,
    "text": "Every operation appears to happen at one"
  },
  {
    "startMs": 559920,
    "text": "point in real time order. Sequential."
  },
  {
    "startMs": 563760,
    "text": "One total order that respects each"
  },
  {
    "startMs": 565920,
    "text": "process own order. Causal"
  },
  {
    "startMs": 569760,
    "text": "operations that are causally related"
  },
  {
    "startMs": 572320,
    "text": "appear in the same order everywhere."
  },
  {
    "startMs": 574720,
    "text": "Session guarantees."
  },
  {
    "startMs": 576880,
    "text": "There are promises to one client like"
  },
  {
    "startMs": 579360,
    "text": "read your rights and monotonic reads"
  },
  {
    "startMs": 582800,
    "text": "eventual. If no new updates are made,"
  },
  {
    "startMs": 586000,
    "text": "reads eventually return the last value."
  },
  {
    "startMs": 589040,
    "text": "A shared context gives you none of"
  },
  {
    "startMs": 590880,
    "text": "these. There's no stated guarantee at"
  },
  {
    "startMs": 593200,
    "text": "all."
  },
  {
    "startMs": 594800,
    "text": "And you don't need the top consistency"
  },
  {
    "startMs": 597440,
    "text": "models."
  },
  {
    "startMs": 599120,
    "text": "The strongest models cannot stay"
  },
  {
    "startMs": 601040,
    "text": "available when the network splits."
  },
  {
    "startMs": 604080,
    "text": "Causal end session guarantees can as"
  },
  {
    "startMs": 606560,
    "text": "long as each clients stick to one place"
  },
  {
    "startMs": 609839,
    "text": "for sub agents that place the"
  },
  {
    "startMs": 612080,
    "text": "orchestrator. So"
  },
  {
    "startMs": 615279,
    "text": "here's a consistency model I use for sub"
  },
  {
    "startMs": 617519,
    "text": "agents. I call it safe handoffs."
  },
  {
    "startMs": 622959,
    "text": "S stands for source feeds."
  },
  {
    "startMs": 626880,
    "text": "Every fact a worker hands on carries its"
  },
  {
    "startMs": 630640,
    "text": "source version as of time and revision"
  },
  {
    "startMs": 634399,
    "text": "and decisions read only fresh source"
  },
  {
    "startMs": 636480,
    "text": "facts."
  },
  {
    "startMs": 638320,
    "text": "A at most once effects"
  },
  {
    "startMs": 641760,
    "text": "every right has an identity and a single"
  },
  {
    "startMs": 644240,
    "text": "use approval for that exact action. An"
  },
  {
    "startMs": 646880,
    "text": "outcome [clears throat] is read back"
  },
  {
    "startMs": 649279,
    "text": "never replayed."
  },
  {
    "startMs": 651440,
    "text": "F fan out budgets. Each downstream gets"
  },
  {
    "startMs": 656000,
    "text": "a gate and a bounded weight and refusal"
  },
  {
    "startMs": 658800,
    "text": "counts as a failure."
  },
  {
    "startMs": 661279,
    "text": "E stands for explicit outcomes."
  },
  {
    "startMs": 664399,
    "text": "Every handoff ends in a named state and"
  },
  {
    "startMs": 667360,
    "text": "a timeout is unknown with its cost kept."
  },
  {
    "startMs": 671279,
    "text": "This consistency model is the one I use"
  },
  {
    "startMs": 674240,
    "text": "for my own sub agents. It's not a"
  },
  {
    "startMs": 675920,
    "text": "universal model. It's based off my own"
  },
  {
    "startMs": 678560,
    "text": "public work. Some of it is in my FCP"
  },
  {
    "startMs": 682000,
    "text": "pacemaker bridge and some are based on"
  },
  {
    "startMs": 684480,
    "text": "my other docs and my own internet draft."
  },
  {
    "startMs": 690640,
    "text": "Now let's see what's behind each letter."
  },
  {
    "startMs": 695920,
    "text": "Yes, source reads."
  },
  {
    "startMs": 698560,
    "text": "It's the answer to that reads where one"
  },
  {
    "startMs": 701519,
    "text": "worker's guess becomes another worker's"
  },
  {
    "startMs": 703360,
    "text": "fact. What helps is to treat every fact"
  },
  {
    "startMs": 707120,
    "text": "as a record, not a sentence."
  },
  {
    "startMs": 709760,
    "text": "Lampot's 1978 paper define when one"
  },
  {
    "startMs": 712880,
    "text": "event happened before another. A fact's"
  },
  {
    "startMs": 715279,
    "text": "provenence is that record. What it was"
  },
  {
    "startMs": 717760,
    "text": "read from and when."
  },
  {
    "startMs": 720240,
    "text": "Let's take this madeup example."
  },
  {
    "startMs": 724320,
    "text": "the record names, the claim and its"
  },
  {
    "startMs": 727200,
    "text": "revision, the source and its version,"
  },
  {
    "startMs": 730560,
    "text": "the scope and three different times,"
  },
  {
    "startMs": 734560,
    "text": "when the data was current, when we read"
  },
  {
    "startMs": 737120,
    "text": "it and when we used it."
  },
  {
    "startMs": 740480,
    "text": "Freshness is checked where the fact is"
  },
  {
    "startMs": 742800,
    "text": "used."
  },
  {
    "startMs": 744639,
    "text": "In this illustration, the data ends at"
  },
  {
    "startMs": 748800,
    "text": "9:05. We read it at 9:06 and we decided"
  },
  {
    "startMs": 753120,
    "text": "at 9:07."
  },
  {
    "startMs": 755200,
    "text": "In this case,"
  },
  {
    "startMs": 757200,
    "text": "we picked a 5 minute limit. So the data"
  },
  {
    "startMs": 761200,
    "text": "decision at 9:07, it's fine. Now at"
  },
  {
    "startMs": 765040,
    "text": "9:15, the second edition came in"
  },
  {
    "startMs": 769040,
    "text": "because our freshness limit is five."
  },
  {
    "startMs": 771279,
    "text": "This is a stale record."
  },
  {
    "startMs": 773760,
    "text": "We reject it"
  },
  {
    "startMs": 776079,
    "text": "and we say that the next station needs"
  },
  {
    "startMs": 777839,
    "text": "another read."
  },
  {
    "startMs": 780079,
    "text": "This is how you separate an observation"
  },
  {
    "startMs": 782320,
    "text": "from an inference. An observation has a"
  },
  {
    "startMs": 784720,
    "text": "source and a read time. An inference has"
  },
  {
    "startMs": 787360,
    "text": "none."
  },
  {
    "startMs": 789279,
    "text": "A most of it is about retries."
  },
  {
    "startMs": 793440,
    "text": "This is an illustration"
  },
  {
    "startMs": 796000,
    "text": "from my article about MCPmaker bridge."
  },
  {
    "startMs": 801279,
    "text": "The orchestrator delegates a task. The"
  },
  {
    "startMs": 804560,
    "text": "sub agent calls a tool which does a real"
  },
  {
    "startMs": 807600,
    "text": "right."
  },
  {
    "startMs": 809279,
    "text": "like"
  },
  {
    "startMs": 810800,
    "text": "creating a PR"
  },
  {
    "startMs": 813200,
    "text": "the reply is lost."
  },
  {
    "startMs": 815600,
    "text": "The orchestrator knows one thing the"
  },
  {
    "startMs": 817920,
    "text": "outcome is unknown."
  },
  {
    "startMs": 820000,
    "text": "So it retries the whole task the right"
  },
  {
    "startMs": 823839,
    "text": "happens again."
  },
  {
    "startMs": 825920,
    "text": "This is a classic problem. In 1984 RPC"
  },
  {
    "startMs": 830079,
    "text": "paper"
  },
  {
    "startMs": 832079,
    "text": "it says that if the call returns"
  },
  {
    "startMs": 834479,
    "text": "[snorts] the procedure ran precisely"
  },
  {
    "startMs": 836000,
    "text": "once. If you get an exception, it ran"
  },
  {
    "startMs": 839040,
    "text": "once or not at all. And you are not told"
  },
  {
    "startMs": 841760,
    "text": "which."
  },
  {
    "startMs": 843600,
    "text": "So you pick a delivery semantic."
  },
  {
    "startMs": 847279,
    "text": "At most once"
  },
  {
    "startMs": 849760,
    "text": "don't retry and accept the fact that it"
  },
  {
    "startMs": 852160,
    "text": "may be lost. At least once. Retry and"
  },
  {
    "startMs": 856800,
    "text": "accept that duplicates are possible."
  },
  {
    "startMs": 860079,
    "text": "Exactly once."
  },
  {
    "startMs": 862320,
    "text": "It happened at least once. And there's a"
  },
  {
    "startMs": 864800,
    "text": "way we can dduplicate"
  },
  {
    "startMs": 868079,
    "text": "maybe by using an item potency key. In"
  },
  {
    "startMs": 871199,
    "text": "MCP item potent hint is only a hint and"
  },
  {
    "startMs": 874399,
    "text": "da or false. If you need it, you build"
  },
  {
    "startMs": 876959,
    "text": "it. Here's a small design from my bridge"
  },
  {
    "startMs": 880880,
    "text": "on how I handle it."
  },
  {
    "startMs": 883440,
    "text": "When a client comes back after restart,"
  },
  {
    "startMs": 886079,
    "text": "the bridge replace the client's record"
  },
  {
    "startMs": 887839,
    "text": "initialize against a fresh server."
  },
  {
    "startMs": 889920,
    "text": "That's safe. It's just a handshake and"
  },
  {
    "startMs": 892240,
    "text": "it does not restore tool state."
  },
  {
    "startMs": 896000,
    "text": "What it never replace is an uncertain"
  },
  {
    "startMs": 898079,
    "text": "tool call. When a shared server is"
  },
  {
    "startMs": 900560,
    "text": "recycled, the bridge drains it and calls"
  },
  {
    "startMs": 903440,
    "text": "till unanswered at the deadline fail as"
  },
  {
    "startMs": 906079,
    "text": "potentially excluded."
  },
  {
    "startMs": 908560,
    "text": "That status is honest. It tells the"
  },
  {
    "startMs": 911360,
    "text": "caller to go and check."
  },
  {
    "startMs": 915199,
    "text": "What helped is giving every side effect"
  },
  {
    "startMs": 917680,
    "text": "an identity. Approval binds to that"
  },
  {
    "startMs": 921440,
    "text": "exact action. It's only used once."
  },
  {
    "startMs": 924880,
    "text": "After the call, read back by that"
  },
  {
    "startMs": 927279,
    "text": "identity."
  },
  {
    "startMs": 928800,
    "text": "Submitted is not confirmed. Confirmed"
  },
  {
    "startMs": 930880,
    "text": "means you observe the effect."
  },
  {
    "startMs": 933680,
    "text": "The last part of a it shows up when work"
  },
  {
    "startMs": 937120,
    "text": "passes and resumes. A lease is a lock"
  },
  {
    "startMs": 940079,
    "text": "with a timeout."
  },
  {
    "startMs": 942079,
    "text": "Martin Clipman's post about distributed"
  },
  {
    "startMs": 944079,
    "text": "locking shows this problem. A client"
  },
  {
    "startMs": 947279,
    "text": "passes while holding it. The lease"
  },
  {
    "startMs": 950399,
    "text": "expense. Someone else takes over and the"
  },
  {
    "startMs": 953440,
    "text": "first client wakes up and writes anyway."
  },
  {
    "startMs": 956800,
    "text": "The fix he proposed is a fencing token."
  },
  {
    "startMs": 960800,
    "text": "A number that goes up every time the"
  },
  {
    "startMs": 962880,
    "text": "lock is granted."
  },
  {
    "startMs": 965120,
    "text": "The storage remembers the highest token"
  },
  {
    "startMs": 967600,
    "text": "it has seen and rejects any right with"
  },
  {
    "startMs": 970800,
    "text": "the old one."
  },
  {
    "startMs": 973120,
    "text": "Sub agents pause and resume all the"
  },
  {
    "startMs": 975600,
    "text": "time. In my other talk, as I put it,"
  },
  {
    "startMs": 980320,
    "text": "restore context, not side effects. So"
  },
  {
    "startMs": 983360,
    "text": "treat an approval like a fencing token."
  },
  {
    "startMs": 985600,
    "text": "One action, one use. Checked at the"
  },
  {
    "startMs": 988079,
    "text": "gate, not remembered in the prompt or"
  },
  {
    "startMs": 990480,
    "text": "conversation."
  },
  {
    "startMs": 993199,
    "text": "My bridge fixed the same class of bug. A"
  },
  {
    "startMs": 996720,
    "text": "stale exit or a deleted resume can no"
  },
  {
    "startMs": 999680,
    "text": "longer remove or bring back a"
  },
  {
    "startMs": 1001040,
    "text": "replacement session."
  },
  {
    "startMs": 1003920,
    "text": "Fan out budgets is about unbounded fan"
  },
  {
    "startMs": 1007440,
    "text": "out. This is flow control or the lack of"
  },
  {
    "startMs": 1010240,
    "text": "it. I hit this issue in my own MCP"
  },
  {
    "startMs": 1013199,
    "text": "bridge before it had a limit on cold"
  },
  {
    "startMs": 1015360,
    "text": "starts."
  },
  {
    "startMs": 1017360,
    "text": "Many sessions start their service at the"
  },
  {
    "startMs": 1020560,
    "text": "same time through npx."
  },
  {
    "startMs": 1023279,
    "text": "They share one package cache. Every one"
  },
  {
    "startMs": 1026480,
    "text": "of them failed with npm's lock"
  },
  {
    "startMs": 1028319,
    "text": "compromised error. Nothing in any single"
  },
  {
    "startMs": 1031439,
    "text": "request was wrong. The failures was the"
  },
  {
    "startMs": 1033520,
    "text": "concurrency itself."
  },
  {
    "startMs": 1035679,
    "text": "And when a downstream does push back, it"
  },
  {
    "startMs": 1038319,
    "text": "can arrive as a plain text. This is MCP"
  },
  {
    "startMs": 1041839,
    "text": "specs one example of a tool execution"
  },
  {
    "startMs": 1043600,
    "text": "error."
  },
  {
    "startMs": 1044720,
    "text": "limit as one sentence. The model reads a"
  },
  {
    "startMs": 1048240,
    "text": "sentence. The orchestrator code may"
  },
  {
    "startMs": 1050320,
    "text": "never look and the next worker starts"
  },
  {
    "startMs": 1052320,
    "text": "anyway."
  },
  {
    "startMs": 1054160,
    "text": "So I introduce a kota budget in my"
  },
  {
    "startMs": 1057200,
    "text": "bridge."
  },
  {
    "startMs": 1058960,
    "text": "Coal starts that use a package manager"
  },
  {
    "startMs": 1062320,
    "text": "go through a gate two at a time by"
  },
  {
    "startMs": 1064720,
    "text": "default. Everyone else waits and the"
  },
  {
    "startMs": 1067919,
    "text": "weight is born at 15 seconds. Past that"
  },
  {
    "startMs": 1071600,
    "text": "the start goes ahead without a gate."
  },
  {
    "startMs": 1074640,
    "text": "And I documented why a corrupted cache"
  },
  {
    "startMs": 1078000,
    "text": "is a risk while a hung client request is"
  },
  {
    "startMs": 1080960,
    "text": "certainity. There is a choice for"
  },
  {
    "startMs": 1083280,
    "text": "availability made on purpose and written"
  },
  {
    "startMs": 1085360,
    "text": "down."
  },
  {
    "startMs": 1087200,
    "text": "And I also built a cap on sessions."
  },
  {
    "startMs": 1090960,
    "text": "At the cap, a new a new session waits up"
  },
  {
    "startMs": 1093919,
    "text": "to 10 seconds and then it's refused."
  },
  {
    "startMs": 1097360,
    "text": "That refusal now counts as a failure in"
  },
  {
    "startMs": 1099200,
    "text": "health. Another book capacity has to be"
  },
  {
    "startMs": 1102640,
    "text": "reserved before the synchronous start"
  },
  {
    "startMs": 1104960,
    "text": "not after otherwise"
  },
  {
    "startMs": 1107919,
    "text": "cued starts or book the cap that is a"
  },
  {
    "startMs": 1110880,
    "text": "check then act race for sub agents give"
  },
  {
    "startMs": 1114559,
    "text": "the orchestrator a number per downstream"
  },
  {
    "startMs": 1117120,
    "text": "a bounded weight and a refused cancel."
  },
  {
    "startMs": 1120720,
    "text": "Now"
  },
  {
    "startMs": 1122240,
    "text": "e the an explicit outcomes every hand of"
  },
  {
    "startMs": 1126559,
    "text": "ends in a named state like stale or"
  },
  {
    "startMs": 1129200,
    "text": "unknown not just text. The last piece is"
  },
  {
    "startMs": 1132559,
    "text": "who decided that state treats agent"
  },
  {
    "startMs": 1135360,
    "text": "output as untrusted input the same way"
  },
  {
    "startMs": 1137840,
    "text": "you treat request body from the"
  },
  {
    "startMs": 1139280,
    "text": "internet."
  },
  {
    "startMs": 1141200,
    "text": "The orchestrator also checks in code."
  },
  {
    "startMs": 1144960,
    "text": "Does the result match the schema? Is the"
  },
  {
    "startMs": 1147200,
    "text": "evidence there? Is it fresh enough? to"
  },
  {
    "startMs": 1149919,
    "text": "the numbers hold when you recmp compute"
  },
  {
    "startMs": 1151760,
    "text": "them. If a check fails, the step returns"
  },
  {
    "startMs": 1155440,
    "text": "a refusal with a reason. It does not"
  },
  {
    "startMs": 1158400,
    "text": "pass the text along with a warning"
  },
  {
    "startMs": 1159840,
    "text": "paragraph underneath. If the same model"
  },
  {
    "startMs": 1162720,
    "text": "can skip a check and declare it past,"
  },
  {
    "startMs": 1165200,
    "text": "you have a request, not a boundary."
  },
  {
    "startMs": 1168240,
    "text": "Here's a madeup example from another"
  },
  {
    "startMs": 1170480,
    "text": "talk. A sub agent says that a rate fell."
  },
  {
    "startMs": 1174960,
    "text": "Before it was 10 errors in,000 requests."
  },
  {
    "startMs": 1178240,
    "text": "after it was six errors with 600"
  },
  {
    "startMs": 1180720,
    "text": "requests. It went from 10 to six and it"
  },
  {
    "startMs": 1183520,
    "text": "claimed that our rate fell. But when we"
  },
  {
    "startMs": 1186480,
    "text": "computed the rate, it's both 1%."
  },
  {
    "startMs": 1190480,
    "text": "We recomputed it and then the claim is"
  },
  {
    "startMs": 1192559,
    "text": "refused."
  },
  {
    "startMs": 1194480,
    "text": "A supported answer is fine. A deliberate"
  },
  {
    "startMs": 1196960,
    "text": "handoff to a person is fine. A confident"
  },
  {
    "startMs": 1199520,
    "text": "guess it's not."
  },
  {
    "startMs": 1201919,
    "text": "So limitations because a consistency"
  },
  {
    "startMs": 1204400,
    "text": "model is only useful when you know where"
  },
  {
    "startMs": 1206320,
    "text": "we should stop."
  },
  {
    "startMs": 1208480,
    "text": "So here's what safe model does not do."
  },
  {
    "startMs": 1214000,
    "text": "It does not make facts true. A source"
  },
  {
    "startMs": 1217039,
    "text": "fact is checkable, not correct. It does"
  },
  {
    "startMs": 1220400,
    "text": "not give you exactly once delivery."
  },
  {
    "startMs": 1223600,
    "text": "A lost reply stays unknown until you"
  },
  {
    "startMs": 1226559,
    "text": "read back. It does not roll anything"
  },
  {
    "startMs": 1229280,
    "text": "back."
  },
  {
    "startMs": 1230880,
    "text": "Undoing a partial task is a compensation"
  },
  {
    "startMs": 1233280,
    "text": "step you design. like the Saga paper"
  },
  {
    "startMs": 1236080,
    "text": "from 1987."
  },
  {
    "startMs": 1238000,
    "text": "And finally, there's no global order"
  },
  {
    "startMs": 1240159,
    "text": "across workers, only causal order per"
  },
  {
    "startMs": 1243120,
    "text": "fact."
  },
  {
    "startMs": 1245039,
    "text": "And it has a price."
  },
  {
    "startMs": 1247840,
    "text": "CAP theorem says that when the network"
  },
  {
    "startMs": 1250159,
    "text": "splits, you trade off availability"
  },
  {
    "startMs": 1253280,
    "text": "against consistency."
  },
  {
    "startMs": 1255600,
    "text": "My bridge chose availability for cold"
  },
  {
    "startMs": 1257919,
    "text": "starts on purpose. For rights, safe"
  },
  {
    "startMs": 1262080,
    "text": "picks consistency over availability. It"
  },
  {
    "startMs": 1266240,
    "text": "refuses rather than guesses."
  },
  {
    "startMs": 1269120,
    "text": "Packlac"
  },
  {
    "startMs": 1270720,
    "text": "adds that even without failures, you"
  },
  {
    "startMs": 1273280,
    "text": "trade latency for consistency."
  },
  {
    "startMs": 1276159,
    "text": "Every freshness check is another read"
  },
  {
    "startMs": 1279120,
    "text": "and and every refusal needs an owner."
  },
  {
    "startMs": 1283280,
    "text": "So where does not fit?"
  },
  {
    "startMs": 1286559,
    "text": "If a sub agent only drafts text that a"
  },
  {
    "startMs": 1289919,
    "text": "person reads before anything happens,"
  },
  {
    "startMs": 1292400,
    "text": "most of this is is a overhead."
  },
  {
    "startMs": 1295039,
    "text": "It also assumes that there's one single"
  },
  {
    "startMs": 1297280,
    "text": "orchestrator."
  },
  {
    "startMs": 1298880,
    "text": "If you have more, this does not fit."
  },
  {
    "startMs": 1302400,
    "text": "Pure agents with no single place to"
  },
  {
    "startMs": 1304720,
    "text": "stick to need something else."
  },
  {
    "startMs": 1307840,
    "text": "This is the consistency model I use for"
  },
  {
    "startMs": 1309760,
    "text": "my own agents that touch actual real"
  },
  {
    "startMs": 1312240,
    "text": "production systems. It's not universal."
  },
  {
    "startMs": 1315679,
    "text": "So wrapping up, here's three key"
  },
  {
    "startMs": 1318320,
    "text": "takeaways."
  },
  {
    "startMs": 1320159,
    "text": "Treat every handoff to a sub agent as a"
  },
  {
    "startMs": 1322480,
    "text": "network call. It can fail halfway. It"
  },
  {
    "startMs": 1325840,
    "text": "can be slow and it can come back twice."
  },
  {
    "startMs": 1328640,
    "text": "Written a status, not just text. If the"
  },
  {
    "startMs": 1331840,
    "text": "result cannot say partial, stale or"
  },
  {
    "startMs": 1333919,
    "text": "unknown, the orchestrator will read it"
  },
  {
    "startMs": 1336080,
    "text": "as done and"
  },
  {
    "startMs": 1339200,
    "text": "start with our safe consistency model"
  },
  {
    "startMs": 1341600,
    "text": "and enforce it in code. Keep what fits"
  },
  {
    "startMs": 1344480,
    "text": "and change what does not. We already"
  },
  {
    "startMs": 1347520,
    "text": "know how to build distribute systems. We"
  },
  {
    "startMs": 1350080,
    "text": "just have to notice that we are building"
  },
  {
    "startMs": 1351600,
    "text": "one."
  },
  {
    "startMs": 1353280,
    "text": "So these are some of the sources behind"
  },
  {
    "startMs": 1355840,
    "text": "this talk."
  },
  {
    "startMs": 1358799,
    "text": "On the left side it's my own work, my"
  },
  {
    "startMs": 1361280,
    "text": "open source projects, my articles,"
  },
  {
    "startMs": 1364559,
    "text": "internet drafts and my talks. On the"
  },
  {
    "startMs": 1367440,
    "text": "right side"
  },
  {
    "startMs": 1369200,
    "text": "specs and standards"
  },
  {
    "startMs": 1371520,
    "text": "and"
  },
  {
    "startMs": 1373039,
    "text": "bunch of papers behind each concept."
  },
  {
    "startMs": 1377120,
    "text": "Thank you."
  }
]
