[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12640,
    "text": ">> This is a clinical note an AI wrote from"
  },
  {
    "startMs": 15200,
    "text": "a real consultation."
  },
  {
    "startMs": 17080,
    "text": "Take a few seconds and read it."
  },
  {
    "startMs": 19320,
    "text": "It reads like a routine headache."
  },
  {
    "startMs": 21600,
    "text": "A new headache, likely tension type,"
  },
  {
    "startMs": 23480,
    "text": "take some paracetamol, come back if it"
  },
  {
    "startMs": 25480,
    "text": "doesn't settle."
  },
  {
    "startMs": 27200,
    "text": "Looks completely fine, doesn't it?"
  },
  {
    "startMs": 30800,
    "text": "Here's what's missing."
  },
  {
    "startMs": 32960,
    "text": "In the room, she also mentioned her jaw"
  },
  {
    "startMs": 35160,
    "text": "aches when she chews. A new headache,"
  },
  {
    "startMs": 37880,
    "text": "over 50 with jaw pain on chewing,"
  },
  {
    "startMs": 40360,
    "text": "that's giant cell arthritis."
  },
  {
    "startMs": 42440,
    "text": "And untreated, it can take her sight"
  },
  {
    "startMs": 43880,
    "text": "within days. It's a same day start"
  },
  {
    "startMs": 46320,
    "text": "steroids now emergency."
  },
  {
    "startMs": 48200,
    "text": "And that one line, it never made it into"
  },
  {
    "startMs": 49640,
    "text": "the note."
  },
  {
    "startMs": 50720,
    "text": "On the page, it's a paracetamol"
  },
  {
    "startMs": 51800,
    "text": "headache."
  },
  {
    "startMs": 52920,
    "text": "And nothing in the note is technically"
  },
  {
    "startMs": 54920,
    "text": "wrong."
  },
  {
    "startMs": 55960,
    "text": "It's the dangerous part is what isn't"
  },
  {
    "startMs": 58280,
    "text": "there."
  },
  {
    "startMs": 59680,
    "text": "And so that's what I'm going to talk"
  },
  {
    "startMs": 60360,
    "text": "about today."
  },
  {
    "startMs": 61600,
    "text": "The dangerous failures are often the"
  },
  {
    "startMs": 63800,
    "text": "ones that actually look completely fine."
  },
  {
    "startMs": 67600,
    "text": "Firstly,"
  },
  {
    "startMs": 68600,
    "text": "who am I? I'm Seb, medical doctor by"
  },
  {
    "startMs": 70600,
    "text": "background, and now I'm Composure, where"
  },
  {
    "startMs": 73480,
    "text": "we build AI evaluation systems for"
  },
  {
    "startMs": 76000,
    "text": "high-stakes domains."
  },
  {
    "startMs": 79800,
    "text": "So, that one was a subtle kind of error,"
  },
  {
    "startMs": 82560,
    "text": "but sometimes it's not subtle at all."
  },
  {
    "startMs": 85400,
    "text": "A man in his 20s sees his GP for a sore"
  },
  {
    "startMs": 87280,
    "text": "throat, tonsillitis."
  },
  {
    "startMs": 89800,
    "text": "The AI writes that up. It gets him chest"
  },
  {
    "startMs": 92480,
    "text": "pain, suspected angina, diabetes"
  },
  {
    "startMs": 94400,
    "text": "medications he's never taken, and an"
  },
  {
    "startMs": 96080,
    "text": "address for a hospital that doesn't"
  },
  {
    "startMs": 97240,
    "text": "exist."
  },
  {
    "startMs": 98600,
    "text": "And I I really like the LLM for this"
  },
  {
    "startMs": 101000,
    "text": "one. I think it's it's a good attempt at"
  },
  {
    "startMs": 102720,
    "text": "hospital name. Um"
  },
  {
    "startMs": 105480,
    "text": "and weeks later, he's invited to"
  },
  {
    "startMs": 106840,
    "text": "diabetic eye screening for diabetes he"
  },
  {
    "startMs": 108440,
    "text": "doesn't have."
  },
  {
    "startMs": 110080,
    "text": "That's genuinely a real case that"
  },
  {
    "startMs": 111400,
    "text": "happened recently."
  },
  {
    "startMs": 114000,
    "text": "Obviously, these kind of crazy ones"
  },
  {
    "startMs": 115600,
    "text": "someone notices, but it's those quiet"
  },
  {
    "startMs": 118360,
    "text": "ones that sit in the record uncalled"
  },
  {
    "startMs": 121240,
    "text": "that are the most challenging"
  },
  {
    "startMs": 123240,
    "text": "and can actually do a lot more damage."
  },
  {
    "startMs": 126920,
    "text": "And they're not rare at all. In the"
  },
  {
    "startMs": 129160,
    "text": "largest real-world study of these notes,"
  },
  {
    "startMs": 131560,
    "text": "about 1 in 20 carried an error that was"
  },
  {
    "startMs": 133840,
    "text": "serious enough that it could cause"
  },
  {
    "startMs": 135120,
    "text": "significant harm to the patient."
  },
  {
    "startMs": 137440,
    "text": "1 in 20. That's not theoretical in"
  },
  {
    "startMs": 139840,
    "text": "testing, that's in production on real"
  },
  {
    "startMs": 141959,
    "text": "patients."
  },
  {
    "startMs": 143160,
    "text": "And that's only the serious the ones. If"
  },
  {
    "startMs": 145400,
    "text": "you widen that lens to all errors,"
  },
  {
    "startMs": 147400,
    "text": "nearly 1 in 5 had an important omission"
  },
  {
    "startMs": 150080,
    "text": "and more than 1 in 10 had a"
  },
  {
    "startMs": 151160,
    "text": "hallucination."
  },
  {
    "startMs": 153760,
    "text": "And AI is being deployed at scale across"
  },
  {
    "startMs": 156280,
    "text": "healthcare fast. Ambient Scribes are one"
  },
  {
    "startMs": 159080,
    "text": "of the leading cases, already in about a"
  },
  {
    "startMs": 161120,
    "text": "third of US practices and climbing."
  },
  {
    "startMs": 163680,
    "text": "Physician AI use doubled last year and"
  },
  {
    "startMs": 166200,
    "text": "none of this is tracked."
  },
  {
    "startMs": 168280,
    "text": "So, for most of these systems, there's"
  },
  {
    "startMs": 169880,
    "text": "no adverse event reporting at all. The"
  },
  {
    "startMs": 172640,
    "text": "errors never show up as incidents, they"
  },
  {
    "startMs": 174480,
    "text": "just sit in the record."
  },
  {
    "startMs": 175880,
    "text": "So,"
  },
  {
    "startMs": 177080,
    "text": "errors this common that are going unseen"
  },
  {
    "startMs": 179239,
    "text": "is is quite hard for me to believe that"
  },
  {
    "startMs": 181480,
    "text": "it's not already affecting patients."
  },
  {
    "startMs": 183720,
    "text": "It's not that we checked and it's fine,"
  },
  {
    "startMs": 185680,
    "text": "it's that we're flying blind."
  },
  {
    "startMs": 189360,
    "text": "And"
  },
  {
    "startMs": 190720,
    "text": "this isn't just a healthcare problem,"
  },
  {
    "startMs": 192400,
    "text": "it's every high-stakes use of AI."
  },
  {
    "startMs": 194680,
    "text": "Healthcare shows it more viscerally"
  },
  {
    "startMs": 196600,
    "text": "because here being confidently wrong"
  },
  {
    "startMs": 198720,
    "text": "can be life and death."
  },
  {
    "startMs": 200400,
    "text": "But, everything I show you can map"
  },
  {
    "startMs": 201680,
    "text": "straight back onto other domains as"
  },
  {
    "startMs": 203280,
    "text": "well."
  },
  {
    "startMs": 205200,
    "text": "So, here's what I want to do. I'm going"
  },
  {
    "startMs": 207200,
    "text": "to show you what exactly is going wrong,"
  },
  {
    "startMs": 209519,
    "text": "why it's going wrong,"
  },
  {
    "startMs": 211400,
    "text": "why the systems we built to catch it"
  },
  {
    "startMs": 212840,
    "text": "don't work, and"
  },
  {
    "startMs": 215000,
    "text": "a suggestion at how maybe we can start"
  },
  {
    "startMs": 217280,
    "text": "to fix that."
  },
  {
    "startMs": 219480,
    "text": "So, first, what's going wrong and why?"
  },
  {
    "startMs": 221760,
    "text": "So,"
  },
  {
    "startMs": 222720,
    "text": "LLMs are getting good, obviously. They"
  },
  {
    "startMs": 224720,
    "text": "don't make stupid mistakes anymore most"
  },
  {
    "startMs": 226400,
    "text": "of the time. So, it's not about dumb"
  },
  {
    "startMs": 227800,
    "text": "errors."
  },
  {
    "startMs": 228800,
    "text": "Everything here came out of three of the"
  },
  {
    "startMs": 230760,
    "text": "best production Ambient scribes on the"
  },
  {
    "startMs": 232880,
    "text": "market."
  },
  {
    "startMs": 234160,
    "text": "Ones that we all know."
  },
  {
    "startMs": 235600,
    "text": "We generated a load of notes across them"
  },
  {
    "startMs": 237320,
    "text": "last week. And this is exactly what's"
  },
  {
    "startMs": 240400,
    "text": "going on right now. This is every"
  },
  {
    "startMs": 242400,
    "text": "failure we found. Each dot is an error"
  },
  {
    "startMs": 244760,
    "text": "colored by type."
  },
  {
    "startMs": 246360,
    "text": "Left to right, how much it matters."
  },
  {
    "startMs": 247959,
    "text": "Bottom to top, whether a strong"
  },
  {
    "startMs": 249320,
    "text": "automated check catches it."
  },
  {
    "startMs": 251680,
    "text": "And that split is the point."
  },
  {
    "startMs": 254440,
    "text": "A handful up top get caught. But almost"
  },
  {
    "startMs": 256840,
    "text": "everything sits below the line. The ones"
  },
  {
    "startMs": 259120,
    "text": "I care about most are these on the"
  },
  {
    "startMs": 261519,
    "text": "bottom right. The high stakes and missed"
  },
  {
    "startMs": 264200,
    "text": "ones."
  },
  {
    "startMs": 266120,
    "text": "Let me show you what a couple of those"
  },
  {
    "startMs": 267400,
    "text": "looks like."
  },
  {
    "startMs": 269919,
    "text": "So, a woman comes in with a headache."
  },
  {
    "startMs": 273000,
    "text": "Doctor asks,"
  },
  {
    "startMs": 274480,
    "text": "\"Did it come on suddenly or build up"
  },
  {
    "startMs": 276080,
    "text": "gradually?\""
  },
  {
    "startMs": 277360,
    "text": "She says she doesn't know. It just"
  },
  {
    "startMs": 278880,
    "text": "happened. The note records that as"
  },
  {
    "startMs": 280800,
    "text": "abrupt sudden onset."
  },
  {
    "startMs": 282760,
    "text": "And sudden onset is a red flag. You can"
  },
  {
    "startMs": 284919,
    "text": "see why it just happened could maybe be"
  },
  {
    "startMs": 288080,
    "text": "interpreted and inferred as abrupt"
  },
  {
    "startMs": 289600,
    "text": "onset."
  },
  {
    "startMs": 290800,
    "text": "But, that's a feature that points to a"
  },
  {
    "startMs": 292800,
    "text": "bleed on the brain. She never said it."
  },
  {
    "startMs": 294560,
    "text": "The model decided it. And now that one"
  },
  {
    "startMs": 296400,
    "text": "word drives the whole workup."
  },
  {
    "startMs": 300240,
    "text": "Here's another."
  },
  {
    "startMs": 301640,
    "text": "Doctor suggests running some tests."
  },
  {
    "startMs": 303600,
    "text": "Patient says, \"Can we just try try"
  },
  {
    "startMs": 305600,
    "text": "antibiotics instead?\""
  },
  {
    "startMs": 307320,
    "text": "They agree, hold off on the tests,"
  },
  {
    "startMs": 309520,
    "text": "treat, and see how it goes."
  },
  {
    "startMs": 311680,
    "text": "Note records the opposite."
  },
  {
    "startMs": 313480,
    "text": "Arrange tests today."
  },
  {
    "startMs": 315320,
    "text": "It kept the plan that they talked out"
  },
  {
    "startMs": 317000,
    "text": "of, not the one they chose. Every line"
  },
  {
    "startMs": 319640,
    "text": "in the note reads fine because it's not"
  },
  {
    "startMs": 322560,
    "text": "really a hallucination at all. It's not"
  },
  {
    "startMs": 324240,
    "text": "wrong. It was there in the original."
  },
  {
    "startMs": 326480,
    "text": "But, it's just not what they ended up"
  },
  {
    "startMs": 327560,
    "text": "deciding."
  },
  {
    "startMs": 330120,
    "text": "So, why are these happening?"
  },
  {
    "startMs": 333720,
    "text": "There's, you know, in ambient scribes,"
  },
  {
    "startMs": 335800,
    "text": "there's first transcription and then"
  },
  {
    "startMs": 337440,
    "text": "generation."
  },
  {
    "startMs": 338800,
    "text": "A lot of it"
  },
  {
    "startMs": 339960,
    "text": "does happen on the transcription layer."
  },
  {
    "startMs": 342400,
    "text": "It can be words misheard for their"
  },
  {
    "startMs": 344760,
    "text": "sound-alikes. So, Humalog heard Humulin."
  },
  {
    "startMs": 348560,
    "text": "Two insulins on completely different"
  },
  {
    "startMs": 350200,
    "text": "timelines, so swapping them could crash"
  },
  {
    "startMs": 352120,
    "text": "a blood sugar."
  },
  {
    "startMs": 353640,
    "text": "Hyperthyroidism becomes hypothyroidism,"
  },
  {
    "startMs": 356280,
    "text": "the opposite condition."
  },
  {
    "startMs": 357840,
    "text": "Or a drop to no"
  },
  {
    "startMs": 360040,
    "text": "on uh no evidence of cancer that becomes"
  },
  {
    "startMs": 363080,
    "text": "evidence of cancer."
  },
  {
    "startMs": 364760,
    "text": "So, these these are really hard"
  },
  {
    "startMs": 365680,
    "text": "problems, and they are common."
  },
  {
    "startMs": 367800,
    "text": "Not the ones I'm going to focus on,"
  },
  {
    "startMs": 369160,
    "text": "because"
  },
  {
    "startMs": 370200,
    "text": "most of what goes wrong is actually even"
  },
  {
    "startMs": 372240,
    "text": "with a perfect transcript."
  },
  {
    "startMs": 374400,
    "text": "It's the model reading the words"
  },
  {
    "startMs": 376160,
    "text": "correctly and still doing one of three"
  },
  {
    "startMs": 378040,
    "text": "things."
  },
  {
    "startMs": 379760,
    "text": "Either it adds something that was never"
  },
  {
    "startMs": 381400,
    "text": "said, it changes something that was, or"
  },
  {
    "startMs": 384000,
    "text": "it omits something that should be there."
  },
  {
    "startMs": 388480,
    "text": "Now, the blatant version of each of"
  },
  {
    "startMs": 390360,
    "text": "these is is really easy to catch. The"
  },
  {
    "startMs": 392160,
    "text": "hard part in all three"
  },
  {
    "startMs": 394800,
    "text": "is"
  },
  {
    "startMs": 396480,
    "text": "the same. It's telling whether that"
  },
  {
    "startMs": 398400,
    "text": "thing that was added or changed or"
  },
  {
    "startMs": 399919,
    "text": "dropped actually matters."
  },
  {
    "startMs": 401720,
    "text": "It's detecting that slight over"
  },
  {
    "startMs": 403440,
    "text": "inference versus the dangerous"
  },
  {
    "startMs": 405080,
    "text": "fabrication. The harmless rephrase"
  },
  {
    "startMs": 407840,
    "text": "versus the meaningful edit."
  },
  {
    "startMs": 409640,
    "text": "A dropped line of small talk versus a"
  },
  {
    "startMs": 411280,
    "text": "dropped allergy."
  },
  {
    "startMs": 413160,
    "text": "So, the ones that matter slip through"
  },
  {
    "startMs": 415880,
    "text": "along with all of the ones that don't."
  },
  {
    "startMs": 419960,
    "text": "That call which different matters"
  },
  {
    "startMs": 422919,
    "text": "is taste, effectively."
  },
  {
    "startMs": 425800,
    "text": "Not aesthetic taste, but essentially"
  },
  {
    "startMs": 428720,
    "text": "judgment. It's It's whether in this"
  },
  {
    "startMs": 430360,
    "text": "context a missed allergy might kill"
  },
  {
    "startMs": 432560,
    "text": "someone or is not important."
  },
  {
    "startMs": 435480,
    "text": "And I think there's there's three"
  },
  {
    "startMs": 436440,
    "text": "properties that really matter about"
  },
  {
    "startMs": 437760,
    "text": "this."
  },
  {
    "startMs": 438800,
    "text": "It's tacit, so your domain experts have"
  },
  {
    "startMs": 441480,
    "text": "it, but they can't fully write it down."
  },
  {
    "startMs": 443760,
    "text": "It's contextual, so the same detail is"
  },
  {
    "startMs": 445800,
    "text": "critical in one note, noise in the next."
  },
  {
    "startMs": 448360,
    "text": "And it's moving. The model changes,"
  },
  {
    "startMs": 450640,
    "text": "guidelines change, two good doctors"
  },
  {
    "startMs": 452520,
    "text": "disagree, different hospitals have"
  },
  {
    "startMs": 454240,
    "text": "different definitions. So, there's no"
  },
  {
    "startMs": 456320,
    "text": "fixed target to write down."
  },
  {
    "startMs": 459600,
    "text": "And so, the model knows the facts,"
  },
  {
    "startMs": 462280,
    "text": "ultimately. They're extraordinarily"
  },
  {
    "startMs": 463560,
    "text": "capable, but what they lack is a sense"
  },
  {
    "startMs": 466040,
    "text": "of what matters"
  },
  {
    "startMs": 467520,
    "text": "here, for this specific example. And"
  },
  {
    "startMs": 469920,
    "text": "that's why even brilliant models make"
  },
  {
    "startMs": 471200,
    "text": "these mistakes."
  },
  {
    "startMs": 473040,
    "text": "So,"
  },
  {
    "startMs": 474080,
    "text": "one natural move, you're never going to"
  },
  {
    "startMs": 476280,
    "text": "make that generator perfect. Generator"
  },
  {
    "startMs": 478520,
    "text": "is cheap. Generation is cheap, so stop"
  },
  {
    "startMs": 480760,
    "text": "fixing it at the source."
  },
  {
    "startMs": 482600,
    "text": "Let it write, put a checker after it,"
  },
  {
    "startMs": 484200,
    "text": "pass only what clears the bar."
  },
  {
    "startMs": 486680,
    "text": "And that checker should be the easier"
  },
  {
    "startMs": 488680,
    "text": "job. The generator has to get everything"
  },
  {
    "startMs": 490640,
    "text": "right and it pay attention to lots of"
  },
  {
    "startMs": 493120,
    "text": "varying instructions."
  },
  {
    "startMs": 495280,
    "text": "Whereas the checker only has to find the"
  },
  {
    "startMs": 496440,
    "text": "one thing that's wrong and just focus on"
  },
  {
    "startMs": 498040,
    "text": "that task. You can also give it more"
  },
  {
    "startMs": 499760,
    "text": "time, more tokens, the exact failure"
  },
  {
    "startMs": 501640,
    "text": "modes to hunt for."
  },
  {
    "startMs": 503400,
    "text": "Evaluation should be easier than"
  },
  {
    "startMs": 504600,
    "text": "generation."
  },
  {
    "startMs": 505880,
    "text": "It's the asymmetry of verification,"
  },
  {
    "startMs": 508440,
    "text": "verifies law. That's why AI is raced"
  },
  {
    "startMs": 510640,
    "text": "ahead anyway, you can cheaply check the"
  },
  {
    "startMs": 512280,
    "text": "answer, maths and code."
  },
  {
    "startMs": 515280,
    "text": "And"
  },
  {
    "startMs": 516520,
    "text": "doing this is exactly what the best"
  },
  {
    "startMs": 518120,
    "text": "teams do. They put a lot of energy into"
  },
  {
    "startMs": 519880,
    "text": "evaluation. It starts with the gold"
  },
  {
    "startMs": 522360,
    "text": "standard, which is expert humans"
  },
  {
    "startMs": 524160,
    "text": "reviewing notes, which obviously works"
  },
  {
    "startMs": 526280,
    "text": "offline, but you can't put a human on"
  },
  {
    "startMs": 528400,
    "text": "every note in production."
  },
  {
    "startMs": 530360,
    "text": "So, they automate it."
  },
  {
    "startMs": 532600,
    "text": "They build a"
  },
  {
    "startMs": 534760,
    "text": "serious system and"
  },
  {
    "startMs": 536720,
    "text": "some of the best versions of this that"
  },
  {
    "startMs": 537960,
    "text": "I've seen are"
  },
  {
    "startMs": 540040,
    "text": "you take the transcript and the note"
  },
  {
    "startMs": 542760,
    "text": "and context, put in front of the judge,"
  },
  {
    "startMs": 545360,
    "text": "a detailed rubric for faithfulness with"
  },
  {
    "startMs": 547720,
    "text": "worked, pass and fail examples."
  },
  {
    "startMs": 550120,
    "text": "The rubric maybe auto-optimized with GPA"
  },
  {
    "startMs": 552480,
    "text": "or something like that. Maybe you have"
  },
  {
    "startMs": 553920,
    "text": "some deterministic NLP to sort of count"
  },
  {
    "startMs": 556160,
    "text": "up medical concepts that are differing"
  },
  {
    "startMs": 558560,
    "text": "between the two."
  },
  {
    "startMs": 560680,
    "text": "That's a powerful system."
  },
  {
    "startMs": 562360,
    "text": "And yet, I pulled all of those errors"
  },
  {
    "startMs": 564600,
    "text": "earlier"
  },
  {
    "startMs": 565760,
    "text": "out of Ambient Scribes in an afternoon."
  },
  {
    "startMs": 569200,
    "text": "So, if the evaluation is this good, how"
  },
  {
    "startMs": 571040,
    "text": "are these errors still getting through?"
  },
  {
    "startMs": 574240,
    "text": "So,"
  },
  {
    "startMs": 575480,
    "text": "I built this system and ran those same"
  },
  {
    "startMs": 578360,
    "text": "notes through it."
  },
  {
    "startMs": 579960,
    "text": "And it scored most of them fine."
  },
  {
    "startMs": 582880,
    "text": "It flagged a handful of them"
  },
  {
    "startMs": 584680,
    "text": "and signed off the rest."
  },
  {
    "startMs": 586886,
    "text": ">> [clears throat]"
  },
  {
    "startMs": 587720,
    "text": ">> But one in five of those clean passes"
  },
  {
    "startMs": 589680,
    "text": "still had some sort of serious error"
  },
  {
    "startMs": 591640,
    "text": "buried in it."
  },
  {
    "startMs": 592880,
    "text": "And often that was an omission."
  },
  {
    "startMs": 595720,
    "text": "The things that should have been there"
  },
  {
    "startMs": 597400,
    "text": "and actually quietly weren't."
  },
  {
    "startMs": 599840,
    "text": "And that's the best version of a judge"
  },
  {
    "startMs": 602480,
    "text": "I've seen in a lot of teams and it waved"
  },
  {
    "startMs": 604800,
    "text": "them through."
  },
  {
    "startMs": 606800,
    "text": "Why did it do that? It's not stupid."
  },
  {
    "startMs": 609600,
    "text": "It's a frontier model, serious"
  },
  {
    "startMs": 610840,
    "text": "engineering behind it, more than clever"
  },
  {
    "startMs": 612480,
    "text": "enough to read the whole encounter and"
  },
  {
    "startMs": 614520,
    "text": "catch every obvious error. And it's not"
  },
  {
    "startMs": 616440,
    "text": "blind, either."
  },
  {
    "startMs": 617880,
    "text": "And and that's part of the trap. If you"
  },
  {
    "startMs": 619520,
    "text": "take a note that says start amoxicillin,"
  },
  {
    "startMs": 621960,
    "text": "when the real decision was actually to"
  },
  {
    "startMs": 623440,
    "text": "wait and see,"
  },
  {
    "startMs": 625400,
    "text": "it's faithful to the words, amoxicillin"
  },
  {
    "startMs": 627480,
    "text": "did come up, but it's a lie about the"
  },
  {
    "startMs": 629240,
    "text": "intent."
  },
  {
    "startMs": 630440,
    "text": "A good judge might catch that, might."
  },
  {
    "startMs": 634200,
    "text": "But whether it flags that versus the"
  },
  {
    "startMs": 636360,
    "text": "other"
  },
  {
    "startMs": 637040,
    "text": "doesn't have other things that it could"
  },
  {
    "startMs": 638240,
    "text": "comment on"
  },
  {
    "startMs": 639560,
    "text": "depends on it knowing what decision"
  },
  {
    "startMs": 641040,
    "text": "matters most."
  },
  {
    "startMs": 642480,
    "text": "And so"
  },
  {
    "startMs": 644200,
    "text": "it's not blind, it just can't tell what"
  },
  {
    "startMs": 646120,
    "text": "counts, essentially."
  },
  {
    "startMs": 648600,
    "text": "So the note passes confidently and you"
  },
  {
    "startMs": 650240,
    "text": "put a judge like that in front of your"
  },
  {
    "startMs": 652080,
    "text": "system, you've not added a safety net,"
  },
  {
    "startMs": 654480,
    "text": "you've added a second silent failure"
  },
  {
    "startMs": 656560,
    "text": "that just nods along with the first."
  },
  {
    "startMs": 659600,
    "text": "And here's the root of it. So in math or"
  },
  {
    "startMs": 662440,
    "text": "code, the verifier comes with free, a"
  },
  {
    "startMs": 665200,
    "text": "unit test, a compiler."
  },
  {
    "startMs": 668440,
    "text": "But for is this note safe and complete,"
  },
  {
    "startMs": 670720,
    "text": "there's no unit test."
  },
  {
    "startMs": 672560,
    "text": "You have to build the verifier yourself"
  },
  {
    "startMs": 674640,
    "text": "and verification is only easier than"
  },
  {
    "startMs": 676920,
    "text": "generation for the easy bit, i.e. spot"
  },
  {
    "startMs": 680440,
    "text": "the difference between transcription"
  },
  {
    "startMs": 681720,
    "text": "note."
  },
  {
    "startMs": 682640,
    "text": "But that's not the hard bit. The hard"
  },
  {
    "startMs": 683560,
    "text": "bit is knowing of all those differences"
  },
  {
    "startMs": 685000,
    "text": "you've seen, which matter. And that's"
  },
  {
    "startMs": 686800,
    "text": "harder than writing that plausibly good"
  },
  {
    "startMs": 688839,
    "text": "general note in the first place."
  },
  {
    "startMs": 691120,
    "text": "Because that standard of good was never"
  },
  {
    "startMs": 693120,
    "text": "written down anywhere that the judge can"
  },
  {
    "startMs": 694720,
    "text": "read it."
  },
  {
    "startMs": 696400,
    "text": "A rubric that you pre-specify is only"
  },
  {
    "startMs": 698920,
    "text": "the taste you could write down."
  },
  {
    "startMs": 700920,
    "text": "The taste that matters is the part that"
  },
  {
    "startMs": 702520,
    "text": "you couldn't."
  },
  {
    "startMs": 703920,
    "text": "And so here's here's a bit more detail"
  },
  {
    "startMs": 705880,
    "text": "on what what matters looks like."
  },
  {
    "startMs": 708960,
    "text": "Two patients, both with blood in their"
  },
  {
    "startMs": 710320,
    "text": "urine, both notes dropped the same kind"
  },
  {
    "startMs": 712800,
    "text": "of line where they'd been on holiday."
  },
  {
    "startMs": 715240,
    "text": "One had been to France, the other to"
  },
  {
    "startMs": 716600,
    "text": "Lake Malawi. Same English emission, same"
  },
  {
    "startMs": 718880,
    "text": "shape, same mistake."
  },
  {
    "startMs": 721960,
    "text": "Well, not really, because blood in the"
  },
  {
    "startMs": 724160,
    "text": "urine obviously warranted away"
  },
  {
    "startMs": 726280,
    "text": "and you're going to have to investigate"
  },
  {
    "startMs": 727360,
    "text": "it, but the France trip is irrelevant."
  },
  {
    "startMs": 730320,
    "text": "The Lake Malawi trip is the diagnosis."
  },
  {
    "startMs": 732840,
    "text": "Fresh water in sub-Saharan Africa means"
  },
  {
    "startMs": 734520,
    "text": "schistosomiasis until proven otherwise"
  },
  {
    "startMs": 737000,
    "text": "and it completely changes what the"
  },
  {
    "startMs": 738200,
    "text": "management plan is. So that same dropped"
  },
  {
    "startMs": 740440,
    "text": "line in one note is pure noise, in the"
  },
  {
    "startMs": 743040,
    "text": "other it's the answer."
  },
  {
    "startMs": 744840,
    "text": "And which one it is, you simply just"
  },
  {
    "startMs": 746000,
    "text": "can't write all of that down in advance."
  },
  {
    "startMs": 749000,
    "text": "So"
  },
  {
    "startMs": 750200,
    "text": "if you can't write it down, you can't"
  },
  {
    "startMs": 751920,
    "text": "write taste down, how do you get that"
  },
  {
    "startMs": 753200,
    "text": "into your evaluator and your whole"
  },
  {
    "startMs": 756120,
    "text": "application system?"
  },
  {
    "startMs": 758120,
    "text": "Well,"
  },
  {
    "startMs": 759040,
    "text": "we've answered a version of this before."
  },
  {
    "startMs": 761400,
    "text": "RLHF exists because you can't write the"
  },
  {
    "startMs": 763120,
    "text": "reward function for good. You learn it"
  },
  {
    "startMs": 765240,
    "text": "from examples by showing it."
  },
  {
    "startMs": 767120,
    "text": "The only question is where you keep what"
  },
  {
    "startMs": 769360,
    "text": "you've learned."
  },
  {
    "startMs": 770839,
    "text": "And there's three places."
  },
  {
    "startMs": 772640,
    "text": "You can either specify it up front, you"
  },
  {
    "startMs": 774839,
    "text": "can"
  },
  {
    "startMs": 775680,
    "text": "stuff the prompt, write the perfect"
  },
  {
    "startMs": 776960,
    "text": "rubric. We just watched that fail"
  },
  {
    "startMs": 779839,
    "text": "essentially."
  },
  {
    "startMs": 781520,
    "text": "You can bake into the weights,"
  },
  {
    "startMs": 783360,
    "text": "fine-tuning or continual learning, but"
  },
  {
    "startMs": 786480,
    "text": "for a standard that's still moving and a"
  },
  {
    "startMs": 787839,
    "text": "score that has to be explainable,"
  },
  {
    "startMs": 789800,
    "text": "the weights, I think, are the wrong"
  },
  {
    "startMs": 791680,
    "text": "place to keep that."
  },
  {
    "startMs": 793560,
    "text": "They go stale, they can't tell you why"
  },
  {
    "startMs": 796240,
    "text": "and"
  },
  {
    "startMs": 797240,
    "text": "you can't change them without a retrain."
  },
  {
    "startMs": 799400,
    "text": "So there's the third option, which I'll"
  },
  {
    "startMs": 801080,
    "text": "show you, which is you essentially just"
  },
  {
    "startMs": 802920,
    "text": "keep the taste as the examples"
  },
  {
    "startMs": 804280,
    "text": "themselves. Past judgments, expert"
  },
  {
    "startMs": 806720,
    "text": "corrections, references, and for each"
  },
  {
    "startMs": 808960,
    "text": "output, you retrieve the ones that bear"
  },
  {
    "startMs": 810720,
    "text": "on it into the judges context,"
  },
  {
    "startMs": 813240,
    "text": "add one and it's live on the next call."
  },
  {
    "startMs": 815280,
    "text": "You can point at exactly what moved the"
  },
  {
    "startMs": 816760,
    "text": "score."
  },
  {
    "startMs": 818400,
    "text": "For this problem, it's both better and"
  },
  {
    "startMs": 820720,
    "text": "also cheaper to do."
  },
  {
    "startMs": 824480,
    "text": "So, that's the way to do that is one"
  },
  {
    "startMs": 826440,
    "text": "repeating loop, three steps."
  },
  {
    "startMs": 828480,
    "text": "Discover the failure modes from real"
  },
  {
    "startMs": 830720,
    "text": "outputs, capture how your experts judge"
  },
  {
    "startMs": 832800,
    "text": "them, calibrate every output against"
  },
  {
    "startMs": 835000,
    "text": "that, and when the standard moves, the"
  },
  {
    "startMs": 836760,
    "text": "loop moves with it."
  },
  {
    "startMs": 839160,
    "text": "So, in more detail, discover. You don't"
  },
  {
    "startMs": 841000,
    "text": "write that rubric in a vacuum."
  },
  {
    "startMs": 843680,
    "text": "You have to put the system in production"
  },
  {
    "startMs": 844800,
    "text": "and look at the real outputs."
  },
  {
    "startMs": 846520,
    "text": "Cluster what goes wrong and the failure"
  },
  {
    "startMs": 847960,
    "text": "modes surface on their own. You name"
  },
  {
    "startMs": 849880,
    "text": "them."
  },
  {
    "startMs": 850760,
    "text": "This is your failure mode ontology."
  },
  {
    "startMs": 852360,
    "text": "Discover from your data, not guess on a"
  },
  {
    "startMs": 854440,
    "text": "whiteboard."
  },
  {
    "startMs": 855760,
    "text": "And you can't shortcut it. The ways that"
  },
  {
    "startMs": 857520,
    "text": "a real system goes wrong are effectively"
  },
  {
    "startMs": 859560,
    "text": "unbounded and synthetic test cases only"
  },
  {
    "startMs": 862600,
    "text": "cover the failures you already imagined."
  },
  {
    "startMs": 865080,
    "text": "The ones that hurt you are often the"
  },
  {
    "startMs": 866640,
    "text": "ones that you didn't. And you'll only"
  },
  {
    "startMs": 868240,
    "text": "find those in real outputs."
  },
  {
    "startMs": 870040,
    "text": "So, this ontology is your map, what to"
  },
  {
    "startMs": 872440,
    "text": "capture judgment on, what to retrieve"
  },
  {
    "startMs": 874240,
    "text": "against, including the failures that you"
  },
  {
    "startMs": 876520,
    "text": "never thought to check for."
  },
  {
    "startMs": 879320,
    "text": "After that, it's capture and then"
  },
  {
    "startMs": 881200,
    "text": "calibrate. So, those discovered modes,"
  },
  {
    "startMs": 884200,
    "text": "they're not a checklist that the judge"
  },
  {
    "startMs": 885600,
    "text": "runs, but they organize everything. What"
  },
  {
    "startMs": 887480,
    "text": "you What you ask your experts about, how"
  },
  {
    "startMs": 889560,
    "text": "you index the cases that you'll"
  },
  {
    "startMs": 890720,
    "text": "retrieve,"
  },
  {
    "startMs": 892200,
    "text": "and"
  },
  {
    "startMs": 893280,
    "text": "capturing is a simple part. You put real"
  },
  {
    "startMs": 895920,
    "text": "outputs in front of your experts."
  },
  {
    "startMs": 897560,
    "text": "Clinicians spend a focused few hours"
  },
  {
    "startMs": 899200,
    "text": "leaving comments. A session doesn't have"
  },
  {
    "startMs": 901680,
    "text": "to be a month-long labeling project to"
  },
  {
    "startMs": 903120,
    "text": "start with. And you collect their"
  },
  {
    "startMs": 904720,
    "text": "judgment."
  },
  {
    "startMs": 906080,
    "text": "Not just a score, but the reasoning and"
  },
  {
    "startMs": 907680,
    "text": "corrections. And over time, you build up"
  },
  {
    "startMs": 909400,
    "text": "that record of how your experts actually"
  },
  {
    "startMs": 911360,
    "text": "judge."
  },
  {
    "startMs": 912640,
    "text": "You then calibrate."
  },
  {
    "startMs": 914040,
    "text": "That's"
  },
  {
    "startMs": 915120,
    "text": "the the the generic part of this you can"
  },
  {
    "startMs": 916720,
    "text": "write down once easily. For example, be"
  },
  {
    "startMs": 918960,
    "text": "faithful or don't drop anything"
  },
  {
    "startMs": 920440,
    "text": "important."
  },
  {
    "startMs": 922000,
    "text": "But what you can't write down is what"
  },
  {
    "startMs": 923320,
    "text": "counts as a serious miss for this"
  },
  {
    "startMs": 925200,
    "text": "specific note. That's contextual. And it"
  },
  {
    "startMs": 927560,
    "text": "shifts from note to note."
  },
  {
    "startMs": 929600,
    "text": "So,"
  },
  {
    "startMs": 931880,
    "text": "what we recommend is you assemble that"
  },
  {
    "startMs": 933360,
    "text": "on the fly. For each output, your"
  },
  {
    "startMs": 935440,
    "text": "judging agent pulls in everything that"
  },
  {
    "startMs": 937120,
    "text": "bears on this one case. It's memory of"
  },
  {
    "startMs": 939200,
    "text": "the most similar outputs"
  },
  {
    "startMs": 941200,
    "text": "that it's judged before and how they"
  },
  {
    "startMs": 942160,
    "text": "scored, the expert corrections that"
  },
  {
    "startMs": 944280,
    "text": "apply, the reference documents and"
  },
  {
    "startMs": 945680,
    "text": "guidelines."
  },
  {
    "startMs": 947880,
    "text": "Just context engineering per output."
  },
  {
    "startMs": 950520,
    "text": "And"
  },
  {
    "startMs": 951640,
    "text": "crucially not just one pre-specified"
  },
  {
    "startMs": 953760,
    "text": "rubric in a vacuum, and not a model that"
  },
  {
    "startMs": 955920,
    "text": "you have to retrain every week,"
  },
  {
    "startMs": 957960,
    "text": "but a full sort of case-specific"
  },
  {
    "startMs": 959880,
    "text": "standard assembled for this output."
  },
  {
    "startMs": 963520,
    "text": "And it's a loop as well."
  },
  {
    "startMs": 965560,
    "text": "Every output you judge, every"
  },
  {
    "startMs": 966520,
    "text": "correction, sharpens the next."
  },
  {
    "startMs": 969120,
    "text": "And when a brand new failure mode"
  },
  {
    "startMs": 970240,
    "text": "appears, Discovery surface it, and it"
  },
  {
    "startMs": 972520,
    "text": "flows straight back in."
  },
  {
    "startMs": 975120,
    "text": "And so, to make that a little bit more"
  },
  {
    "startMs": 975920,
    "text": "concrete,"
  },
  {
    "startMs": 977720,
    "text": "that headache that I opened with, the"
  },
  {
    "startMs": 979560,
    "text": "one that was really a possible blindness"
  },
  {
    "startMs": 980720,
    "text": "emergency,"
  },
  {
    "startMs": 982120,
    "text": "here's the kinds of things that you"
  },
  {
    "startMs": 983080,
    "text": "would want to pull in for that note."
  },
  {
    "startMs": 985839,
    "text": "The nearest cases that your experts have"
  },
  {
    "startMs": 987520,
    "text": "judged,"
  },
  {
    "startMs": 988880,
    "text": "not this exact patient, but the same"
  },
  {
    "startMs": 990480,
    "text": "shape, maybe a red flag filed as"
  },
  {
    "startMs": 992120,
    "text": "routine."
  },
  {
    "startMs": 993320,
    "text": "Uh the corrections that apply, like a"
  },
  {
    "startMs": 995520,
    "text": "new headache over 50, um"
  },
  {
    "startMs": 998960,
    "text": "suggests something that you need to"
  },
  {
    "startMs": 1000000,
    "text": "check red flags on, and some criteria"
  },
  {
    "startMs": 1003000,
    "text": "and guidelines, and you pull all of that"
  },
  {
    "startMs": 1005080,
    "text": "in."
  },
  {
    "startMs": 1006720,
    "text": "It hasn't memorized this case. It's a"
  },
  {
    "startMs": 1008440,
    "text": "capable model."
  },
  {
    "startMs": 1010040,
    "text": "And handed the right context to reason"
  },
  {
    "startMs": 1011440,
    "text": "from, held against that, the dropped red"
  },
  {
    "startMs": 1013600,
    "text": "flag stands out. It was never actually"
  },
  {
    "startMs": 1015520,
    "text": "hard to catch. It just didn't know what"
  },
  {
    "startMs": 1017560,
    "text": "mattered."
  },
  {
    "startMs": 1019200,
    "text": "And so, if you take that same data set"
  },
  {
    "startMs": 1020520,
    "text": "generated notes from the start and pass"
  },
  {
    "startMs": 1022320,
    "text": "it through these three"
  },
  {
    "startMs": 1024199,
    "text": "judging systems,"
  },
  {
    "startMs": 1026280,
    "text": "the first, a strong off-the-shelf"
  },
  {
    "startMs": 1029240,
    "text": "judge with a rubric frontier model,"
  },
  {
    "startMs": 1031839,
    "text": "um"
  },
  {
    "startMs": 1033319,
    "text": "it's better than a coin flip, but it"
  },
  {
    "startMs": 1034600,
    "text": "misses most of what matters."
  },
  {
    "startMs": 1037079,
    "text": "The second, that sort of serious system"
  },
  {
    "startMs": 1038600,
    "text": "that we talked about before,"
  },
  {
    "startMs": 1040640,
    "text": "rubric, deeper,"
  },
  {
    "startMs": 1042880,
    "text": "maybe some detona stick checks, better"
  },
  {
    "startMs": 1044920,
    "text": "again,"
  },
  {
    "startMs": 1046120,
    "text": "but still missing quite a lot of what"
  },
  {
    "startMs": 1047680,
    "text": "counts."
  },
  {
    "startMs": 1048920,
    "text": "The third, the judge running this loop,"
  },
  {
    "startMs": 1050960,
    "text": "discovered failure modes, calibrated for"
  },
  {
    "startMs": 1052520,
    "text": "output against what experts judged,"
  },
  {
    "startMs": 1055480,
    "text": "is"
  },
  {
    "startMs": 1056520,
    "text": "performing a lot better on this specific"
  },
  {
    "startMs": 1058720,
    "text": "data set."
  },
  {
    "startMs": 1059840,
    "text": "Same notes."
  },
  {
    "startMs": 1061200,
    "text": "The only thing that changes is what the"
  },
  {
    "startMs": 1062200,
    "text": "judge was shown."
  },
  {
    "startMs": 1063680,
    "text": "And the difference here, it's not more"
  },
  {
    "startMs": 1064800,
    "text": "compute or a better prompt, it's that"
  },
  {
    "startMs": 1066840,
    "text": "the first two fight taste and lose. They"
  },
  {
    "startMs": 1068680,
    "text": "guess the criteria, they freeze one"
  },
  {
    "startMs": 1070800,
    "text": "standard, and they go stale."
  },
  {
    "startMs": 1073160,
    "text": "This repeating evolving loop does the"
  },
  {
    "startMs": 1075120,
    "text": "opposite. It discovers the modes, fits"
  },
  {
    "startMs": 1077680,
    "text": "the standard to each mode, and keeps"
  },
  {
    "startMs": 1079520,
    "text": "learning."
  },
  {
    "startMs": 1081920,
    "text": "So, you might not write chemical notes,"
  },
  {
    "startMs": 1084840,
    "text": "but if you ship anything where being"
  },
  {
    "startMs": 1086680,
    "text": "confidently wrong has a cost, the"
  },
  {
    "startMs": 1088680,
    "text": "contract review that misses the clauses"
  },
  {
    "startMs": 1091080,
    "text": "that change the deal, the support agent"
  },
  {
    "startMs": 1092960,
    "text": "that promises a refund you don't offer,"
  },
  {
    "startMs": 1096480,
    "text": "the same thing is true for all of those."
  },
  {
    "startMs": 1097960,
    "text": "It's watched, if at all, by a judge with"
  },
  {
    "startMs": 1101480,
    "text": "no taste for what matters in your"
  },
  {
    "startMs": 1102840,
    "text": "domain. So,"
  },
  {
    "startMs": 1105240,
    "text": "three things. Discover your failure"
  },
  {
    "startMs": 1106840,
    "text": "modes from real outputs, don't guess"
  },
  {
    "startMs": 1108880,
    "text": "them. Capture your experts' judgment on"
  },
  {
    "startMs": 1111600,
    "text": "them, the standard that they can't write"
  },
  {
    "startMs": 1113120,
    "text": "down."
  },
  {
    "startMs": 1114200,
    "text": "Calibrate every output against the cases"
  },
  {
    "startMs": 1116280,
    "text": "that they've already judged, not a"
  },
  {
    "startMs": 1118040,
    "text": "static rubric, not a retrained model."
  },
  {
    "startMs": 1120920,
    "text": "Then keep that loop running."
  },
  {
    "startMs": 1122920,
    "text": "And if you take one thing away,"
  },
  {
    "startMs": 1124800,
    "text": "easiest place to start is your experts"
  },
  {
    "startMs": 1127160,
    "text": "leaving free-form comments on real"
  },
  {
    "startMs": 1129120,
    "text": "outputs."
  },
  {
    "startMs": 1130400,
    "text": "That's the real That's the raw material"
  },
  {
    "startMs": 1132520,
    "text": "for everything else."
  },
  {
    "startMs": 1134920,
    "text": "Your judge can verify anything that you"
  },
  {
    "startMs": 1137080,
    "text": "write down in advance, but the standard"
  },
  {
    "startMs": 1139320,
    "text": "of good never could be. And so, stop"
  },
  {
    "startMs": 1141520,
    "text": "trying to write it all down in advance,"
  },
  {
    "startMs": 1143600,
    "text": "and just start capturing it case by"
  },
  {
    "startMs": 1145000,
    "text": "case, and evolving it."
  },
  {
    "startMs": 1147080,
    "text": "That's why evaluation can't be a thing"
  },
  {
    "startMs": 1149640,
    "text": "you build once and freeze."
  },
  {
    "startMs": 1151720,
    "text": "The standard it checks against doesn't"
  },
  {
    "startMs": 1154120,
    "text": "exist on paper."
  },
  {
    "startMs": 1155600,
    "text": "It has to be discovered from real"
  },
  {
    "startMs": 1156560,
    "text": "outputs captured from the people who"
  },
  {
    "startMs": 1158520,
    "text": "hold it and kept alive as it moves."
  },
  {
    "startMs": 1162000,
    "text": "Evaluation isn't something you have,"
  },
  {
    "startMs": 1163680,
    "text": "it's something that you do continuously"
  },
  {
    "startMs": 1165800,
    "text": "over time."
  },
  {
    "startMs": 1167240,
    "text": "Thank you."
  },
  {
    "startMs": 1169188,
    "text": ">> [applause]"
  }
]
