Contents
  1. Purpose and development
    1. Healthcare work and settings
    2. Intended contribution
    3. Continuing lines of development
  2. Applications and completed work
    1. Documentation and total work
    2. Administration and access
    3. Detection and clinical response
  3. Records, exchange, and permitted use
    1. The record as an account of care
    2. Preserve meaning through FHIR
    3. Access and permitted use
  4. Patient-specific conclusions
    1. Applicability beyond relevance
  5. Responsibility and integration
    1. Meaningful clinical review
    2. Accepted clinical handoffs
    3. Commitment at the point of care
  6. Evidence for use
    1. Validate the task and population
    2. Errors, probabilities, and workload
    3. Evaluate the complete intervention
  7. Operation and recovery
    1. Monitor changing conditions
    2. Recovery and affected work
  8. Check understanding
  9. Open questions
  10. Selected talks
  11. References
  12. Talk library
← All topics

AI in healthcare

AI can help healthcare teams prepare documentation, find relevant information, identify concerns, and coordinate work. The engineering task is to connect that capability to a service people can actually use. A fluent note, an accurate prediction, a successful record exchange, and an accepted handoff each accomplish something different. This chapter follows those distinctions from intended use through integration, evaluation, and recovery.

I. Purpose and development

Healthcare work and settings

Healthcare involves assessing people, delivering care, documenting what happened, and coordinating what happens next. A clinical workflow comprises the physical and mental tasks people perform within and across care environments. Some tasks occur together; others wait for information or another person. Care coordination organizes those responsibilities and communications across participants. Improving one task helps only if the surrounding work still reaches its intended result.

A care setting is the environment in which care occurs. A patient population is the group an application is intended to serve. These definitions become requirements: which people can use the interface, what information is available, and who can respond when assistance is insufficient. The following comparison identifies design considerations, not universal staffing arrangements or response deadlines.

SettingWork and informationResponsibility to establish
Outpatient visitsCare without hospital admission; preparation, consultation, and later follow-up may occur separately.Ownership of results and unfinished work after the visit.
Inpatient careCare during hospital admission; observations and assessments accumulate across a team.Who reviews a concern and who assesses the patient.
Emergency accessTriage prioritizes attention by urgency rather than supplying a complete diagnosis.The route to an appropriately urgent clinical response.
Remote follow-upInformation arrives through a mediated interaction rather than an in-person examination.Eligibility for the interaction and access to clinician-delivered care.

Population limits can be concrete. A 2024 prospective study of the Dora R1 telephone assistant examined follow-up after routine cataract surgery, with an ophthalmologist supervising calls in real time. It excluded non-English speakers and people with hearing or cognitive difficulties. That study therefore cannot establish suitability for everyone who might receive a telephone call. Likewise, domain expertise must match the task: experience as a physician does not necessarily include experience performing medical coding.

Intended contribution

Intended use connects a specific task to its users, patient population, setting, available information, output, permitted actions, and exclusions. Start there rather than with a model or an agent framework. Clinical decision support supplies information at useful points in care to assist decisions. It can take the form of reference information, reminders, summaries, templates, or rules; it need not involve AI. The existing workflow or a simpler rule is therefore a meaningful alternative.

A learned prediction is an estimate produced by behavior fitted from examples rather than entirely authored rules; Machine Learning Fundamentals explains that distinction. A generated answer is a composed response, not necessarily a passage retrieved intact. In retrieval-augmented generation, selected records or references supply information for that composition. Neither mechanism determines who may act on the output.

For example, a bounded result-review assistant might prepare a source-linked summary for authorized clinicians using records available for the selected patient and encounter. Its output remains a draft; missing required information is surfaced for review; it cannot independently place orders. This specification identifies both useful work and a limit on delegation. Displaying a concern, moving it into a review queue, and committing a change to a record are separate operations, even when they originate from the same model output.

Medical-purpose terminology also follows intended use rather than product branding. The International Medical Device Regulators Forum’s Software as a Medical Device definition concerns software performing an intended medical purpose without being part of a hardware medical device. Calling an application a copilot does not settle its local regulatory status, risk class, or authorization requirements.

Continuing lines of development

Healthcare AI developed along several lines because reasoning, information delivery, pattern recognition, and documentation solve different problems. These turning points explain the continuing division of work.

LandmarkContribution
Ledley–Lusted framework — July 3, 1959Robert Ledley and Lee Lusted’s diagnostic-reasoning paper separated logic, probability, and treatment value judgments: foundations for assistance, not a deployed diagnostic service.
HELP — around 1970; 1975At LDS Hospital and the University of Utah, a common clinical database and then a medical-decision language connected decision support to incoming patient information.
MYCIN — 1972 beginnings; 1974 implementationStanford’s rule-based consultation project, implemented in Edward Shortliffe’s dissertation, made expert knowledge explicit, inspectable, and modifiable.
Learned retinal-image detection — November 29, 2016Varun Gulshan, Lily Peng, and collaborators demonstrated detection from learned visual features, leaving clinical use and patient benefit as further questions.

Contemporary ambient documentation applies language assistance to producing records for later care and billing. It addresses another persistent task rather than replacing diagnosis or hospital decision support. Explicit rules, timely information, learned detection, and documentation assistance remain useful for different parts of healthcare work.

II. Applications and completed work

Documentation and total work

A clinical note records information about care, such as the history discussed, findings, assessment, and plan. It becomes part of the electronic health record, or EHR: the digital record accumulated across care interactions. Ambient documentation captures a care conversation and generates a draft for professional checking and correction. The task ends with usable documentation, not with generated text.

Checking must cover both unsupported additions and consequential omissions. A sentence can sound plausible while introducing something the encounter did not establish; a concise draft can omit information needed by the next clinician. Notes also support billing, so errors can travel into both clinical and administrative work. From Ambient Documentation to Clinical Intelligence explains these downstream uses.

Inspectability can reduce the distance between a disputed sentence and its evidence. Abridge’s linked-evidence interface connects selected note text to transcript passages and original audio. The transcript and note remain derived representations: a link helps a clinician inspect the source but does not establish accurate speech recognition or valid clinical interpretation.

This fictional encounter separates generation of a transcript and an unfinalized draft from reviewer navigation back to the same source segment. Source links enable checking; they do not certify transcription or interpretation.

A 24-week randomized trial involving 66 ambulatory practitioners in one health system introduced the documentation-only tool at staggered, randomly assigned stages—a stepped-wedge design. Compared with usual practice, it reported less time spent on notes and lower work exhaustion/interpersonal disengagement. Participants were voluntary early adopters and knew when they were using the tool. Unedited-note quality was assessed by a model judge, an AI model scoring the notes, rather than comprehensive independent clinical review. These findings concern documentation and practitioner outcomes, not improved diagnosis or patient health.

Administration and access

Administrative assistance organizes access and payment rather than determining a diagnosis. A referral requests evaluation or treatment from another clinician or service. Intake assembles the clinical question, relevant history, and records for specialty review. The reviewer may advise, schedule care, or request workup—additional information or investigations needed to assess the patient. The referring team obtains missing material, watches for requested results, and resubmits the referral for review. When a visit is needed, scheduling and follow-up must remain tracked: an appointment does not establish attendance or completed care.

Within this process, a proposed AI intake assistant could organize supplied records and flag missing requested material for the referring team. Its value should be compared with the existing workflow. An AHRQ-supported non-AI referral evaluation found shorter initial waits in most studied medical clinics, but interviews identified shifted work and appointment-selection difficulties. Faster intake alone would not establish better access to completed care.

Payment work has similar boundaries. A payer is the organization financing covered care; prior authorization is a process for reviewing a proposed service against coverage requirements. A denial appeal seeks reconsideration of a rejected request or claim. Preparing a clinical appeal requires reconciling patient records, care guidelines, and payer policies under a deadline—not merely generating a persuasive letter. In the announced Anterior–Stellarus workflow, AI assists research and policy application while licensed clinicians make final determinations. That division of work does not itself demonstrate better access.

Detection and clinical response

Screening looks for a target condition in an intended population. Diagnosis determines what condition is present. Prognosis estimates a future outcome. Triage prioritizes attention by urgency, also called acuity. These tasks produce different outputs, but all require clarity about the response that follows. An urgency category is not a complete diagnosis, and identifying risk does not establish which intervention will help.

Hospital deterioration monitoring makes the response dependency visible. Kaiser Permanente describes Advance Alert Monitor as a coordinated service: threshold crossings prompt off-site nurse review, a reviewing nurse contacts the bedside rapid-response nurse, and bedside assessment informs a care-team decision that considers patient preferences. This operating account explains responsibilities; it is not a model-only intervention or, by itself, a causal estimate of patient benefit.

Outpatient screening must connect image interpretation to care beyond the screening visit. Gulshan, Peng, and collaborators’ 2016 retinal-image study demonstrated detection using learned features rather than explicitly programmed lesion detectors. It tested different operating thresholds against ophthalmologist judgments and called for clinical-setting and patient-outcome studies. IDx-DR’s 2018 pivotal study, led by Michael Abràmoff and colleagues, examined a bounded diagnostic workflow in ten primary-care offices. Trained office staff acquired images, and the system supplied image-quality guidance and a diagnostic result without specialist interpretation of every screening image. This prospective study tested more than classification of an existing image collection, but did not establish completed follow-up or prevention of vision loss.

ACCESS, reported by Risa Wolf and colleagues in 2024, separates screening from subsequent care. At two pediatric diabetes sites, point-of-care AI screening at the visit was compared with referral and scripted education. Follow-up among intervention participants with positive results used a different denominator from screening in either arm. Screening completion and indicated follow-up are different outcomes.

Screening completion and indicated follow-up

ACCESS: youth with diabetes at two pediatric sites. Positive-result follow-up belongs to a subgroup of the intervention cohort.

Screening and follow-up have different denominators81 of 81 intervention participants screened at the visit, including 25 with positive results. Of those 25, 16 attended eye care within six months. Separately,18 of 82 analyzed controls completed screening within six months.One participant scale for every bar; each denominator is stated explicitly.Intervention · screening at the visit81/81 screened25 positive results within the 81 screenedFollow this subgroup onlyPositive-result subgroup · eye-care attendance within six months16/25 attended9 without recorded attendance in this window; this does not establish disease absence.Separate analyzed controls · screening within six months18/82 screenedReferral and scripted education; this is not a positive-result follow-up rate.
ACCESS studied youth with diabetes; the device was not labeled for this pediatric population, and specialists overread all study images. Screening, eye-care attendance, completed treatment and preservation of vision are different endpoints; the study did not measure vision preservation.

More generally, high risk does not imply large treatment benefit: an adverse outcome might remain likely with or without the proposed intervention. Prediction and causal effect are separate claims. Patient-facing assistance needs the same discipline. A remote follow-up conversation can collect information and request clinical involvement, but nuanced symptoms and access to clinician-delivered care remain part of the task.

III. Records, exchange, and permitted use

The record as an account of care

An EHR contains accounts of observations and care, not a complete state of the patient. An encounter is a particular interaction with a healthcare service. Patient identity establishes whose information is represented; encounter identity distinguishes which care interaction it concerns. Notes describe events and interpretations, orders request activities, and results report findings. Combining them without preserving those roles changes their meaning.

Medication records illustrate the difference. A prescription contains an order and instructions; dispensing records a supplied quantity; administration records that medicine was given; a medication statement reports that it was taken or given. A prescription is not proof of administration. Similarly, a statement that a symptom was denied, a statement that it occurred previously, and no statement about it are not interchangeable.

Provenance records where information came from and how it was produced or revised. Preserve it alongside the information; Data Quality and Curation explains the general principle. For a clinical result, distinguish when the observation applies, when a result version became available to providers, and when the application received it. A result collected at 09:00 but issued at 10:15 cannot support a prediction made at 09:30. Later corrections must not silently replace the historical account of what was available.

Collection, issue, receipt and retained snapshots

One investigation, with example timestamps. All three application snapshots remain available as historical records.

A receipt enables its own decision snapshotCollected 09:00. Version 1 issued 10:15 and received 10:20. Correction version 2 received 11:05, issue time unspecified. Snapshot 09:30 has no result; snapshot 10:20 version 1; snapshot 11:05 version 2. Earlier snapshots are retained.Clinical eventProvider issueApplication receipt09:00 collected10:15 version 110:20 version 111:05 version 2Preserved snapshots09:30No result available10:20Version 1 retained11:05Version 2 availableHorizontal position follows time. Correction issue time is unspecified. Collection alone enables no result snapshot.
Collection, provider issue and application receipt are different events. Arrows connect receipts only to the snapshots they enable. The correction’s issue time is unspecified; its arrival changes new work without changing what earlier decisions knew.

Status and absence also carry information. A report may be preliminary, final, or corrected. A missing measurement might be unperformed, not asked about, temporarily unknown, withheld, or lost through an error. FHIR supplies distinct data-absent-reason codes for such cases. If no reason is recorded, the application cannot invent one. Information may also exist only in narrative text or another institution’s records; an absent smoking-history code, for example, does not identify a nonsmoker.

Preserve meaning through FHIR

Interoperability means exchanging information that recipients can use with its intended meaning. Fast Healthcare Interoperability Resources, or FHIR, provides typed building blocks called resources, connected through references. A profile constrains a resource for a particular use—for example, by specifying required content or terminology. The FHIR R5 overview describes these mechanisms. Actual integrations must establish their deployed version, profiles, and supported behavior; shared syntax alone does not establish complete or clinically consistent records.

Consider a result-review application. A ServiceRequest describes a requested investigation. A DiagnosticReport can reference that request, its encounter, and Observation resources containing individual results. Patient references identify whose information is represented. These are links between distinct records, not interchangeable copies of one document. A review draft should retain its source-result version so that the basis of the summary remains inspectable.

A result retains its context

Example

A review draft derives from a result version, while resource references preserve the investigation and patient context.

ServiceRequestRequested investigation.
DiagnosticReportReport connecting the investigation and its results.
Observation, selected versionIdentified result version with applicable time, status and typed content.
PatientWhose information is represented.
EncounterThe particular care interaction.
Review draftApplication draft retaining its source-version identity.
Example resource links, not universally mandatory fields. Solid reference arrows point to the referenced resource; the dashed derivation arrow points from the selected result version to its application draft. The draft is not a required FHIR resource type.
Read the diagram as text
  • ServiceRequest. Requested investigation.
  • DiagnosticReport. Report connecting the investigation and its results.
  • Observation, selected version. Identified result version with applicable time, status and typed content.
  • Patient. Whose information is represented.
  • Encounter. The particular care interaction.
  • Review draft. Application draft retaining its source-version identity.
  • DiagnosticReport → ServiceRequest: Reference: basedOn.
  • DiagnosticReport → Encounter: Reference: encounter.
  • DiagnosticReport → Patient: Reference: subject.
  • DiagnosticReport → Observation, selected version: Reference: result.
  • Observation, selected version → Patient: Reference: subject.
  • Observation, selected version → Review draft: Derivation: summarized into.

An Observation separates what was observed from its result and optional interpretation. Results can use different datatypes or appear in components; a missing top-level number is not a normal finding. LOINC, Logical Observation Identifiers Names and Codes, supplies shared identifiers for observations. Maintained by the Regenstrief Institute, it helps recipients interpret tests originally named by local codes. LOINC identifies the observation; FHIR transports its representation.

Preserve a code’s terminology system and version; matching display labels do not establish equivalent meanings. UCUM, the Unified Code for Units of Measure, supplies coded units that can support compatible conversions. Preserve comparators too: a result below a value is not exactly that value. Matching units alone does not establish the same measurement, specimen, or method. The FHIR datatype definitions make these representation distinctions explicit.

SMART on FHIR adds application-launch and authorization capabilities. An EHR launch can provide selected patient and encounter context, while declared server capabilities describe what is supported. This is separate from the resource representation. Existing HL7 Version 2 feeds exchange messages for activities such as laboratory orders and patient transfers. Converting them to FHIR still requires local mapping because implementations differ in field use and placement. A new API does not eliminate that interpretation work.

Access and permitted use

Purpose limitation constrains processing to an approved purpose. Data minimization limits information, recipients, and retention to what that purpose requires. Secondary use means another use, such as developing models from material collected for care. Reading an EHR, recording a conversation, sending it to a service, retaining a draft, and reusing it for development require separate consideration. Approve actual service paths explains why the product, feature, endpoint, recipients, and settings matter more than a provider name.

For each artifact, record the approved purpose, recipients, permission basis, retention rule, and accountable owner. Do not assign one lifetime to the entire encounter.
ArtifactDecision to document
Source recordWhich patient data and operations the application may access.
Encounter audioRecording arrangements, processing recipients, and disposition.
Draft noteAuthorized reviewers, record-entry process, and draft retention.
Service requestFields sent, actual processing path, logs, and reuse terms.
Audit entryAction, actor, authority, source/version references, and protected evidence access.

NHS England’s ambient-scribing guidance calls for explaining the tool and respecting patient objections. In that setting, retain a workable documentation route without optional recording. This is scoped guidance, not a universal consent rule.

Removing obvious identifiers does not settle reuse or eliminate identification risk. HHS guidance describes HIPAA’s Safe Harbor and Expert Determination methods, including identifiers embedded in free text, and acknowledges residual risk even after proper de-identification. A name-stripping function is therefore not a compliance determination. Similarly, useful auditability does not require copying every sensitive payload into ordinary logs: preserve attributable actions and references, with evidence access and retention governed separately.

IV. Patient-specific conclusions

Applicability beyond relevance

A clinical guideline offers recommendations for specified circumstances. Finding a relevant passage is not the same as establishing that its circumstances hold for this patient. Evidence sufficiency concerns whether the available information permits the requested conclusion. Here that requires matching patient identity, current status, relevant history, exclusions, source dates, and local use conditions.

The HL7 clinical-practice-guideline guide separates describing a patient from proposing an activity. It also separates eligibility for a pathway from enrollment: preferences or other clinical considerations can prevent participation even when eligibility holds. An application should therefore keep required premises visible instead of compressing a guideline and a record into an unexplained yes.

This schematic comparison illustrates evidence states without specifying medical eligibility criteria.
Premise neededRecord evidencePermitted interpretation
A particular investigation has a resultA linked result is present.Summarize that result with its status and source.
The result has the required final statusThe available report is preliminary.The final-status requirement is not established.
A relevant historical condition is absentThat history is undocumented.The condition remains unknown, not absent.

Clinical meaning can fail even when terminology matches. In Lovejoy’s prior-authorization example, an answer treated imaging findings as establishing suspected disease, while the speaker interpreted the existing confirmed diagnosis as changing the guideline answer. The lesson is to evaluate the actual criterion with the history, not to generalize that interpretation to every use of the word suspected. Source identity matters too: a patient assertion and a clinician’s documented finding should not become indistinguishable merely because both entered a summary.

Missing premises can guide the next question. MYCIN worked backward from a goal through expert-authored rules and asked for facts it could not derive. Its explanation subsystem exposed supporting rules and reasons for questions. Its certainty factors represented degrees of belief, not conditional probabilities. The medical knowledge was separate from the execution machinery: removing that knowledge base later produced EMYCIN, a framework for building expert systems in other domains. Neither an inspectable rule trace nor a modern source-linked answer establishes that the underlying medical interpretation is correct. Abstention withholds an unsupported claim; it can still preserve a useful factual summary and identify the information or clinical review required before answering further.

V. Responsibility and integration

Meaningful clinical review

Review is work, not a safety property conferred by an approval button. A reviewer needs suitable expertise, the original evidence, time to inspect it, and authority to reject the output. Expertise should match the workflow and its failure modes. The person operating the software, the person assessing clinical content, and the person authorized to change care need not be the same person.

Assign these responsibilities explicitly, even when one person holds several roles.
RoleResponsibility
Service operatorMaintain functioning interfaces, access controls, and incident response.
Clinical assessorEvaluate content against patient evidence and the clinical task.
Action authorizerApprove the particular consequential operation within their authority.
Unresolved-work ownerEnsure pending review or follow-up receives an accepted disposition.

Automation bias is inappropriate reliance on automated advice. In a web-based experiment with 223 dermatologists, participants assessed 24 cases before and after AI advice, including five deliberately incorrect recommendations. Some initially correct answers became wrong after erroneous advice; useful advice was also rejected. This controlled task did not measure patient outcomes, but it shows why oversight assessment must distinguish accepting helpful advice from rejecting harmful advice.

Alert fatigue concerns reduced responsiveness associated with burdensome alerting. A retrospective primary-care study associated more reminders per encounter and repeated reminders for the same patient with lower acceptance. It did not establish a general workload effect or assess whether overrides were appropriate. Lowering override rates is therefore not automatically a safety improvement.

Make corrections diagnostically useful. A review surface can place the source record, guideline, and requested conclusion together, then collect why an answer is wrong rather than only an incorrect label. Such critiques help distinguish missing evidence, misunderstood criteria, and unsupported additions. Review before consequential commitment develops the interface mechanics. Patients and staff also need a route to report disputed content to an accountable reviewer and learn its disposition.

Accepted clinical handoffs

A clinical handoff transfers information, responsibility, and authority. Under AHRQ TeamSTEPPS guidance, the sender remains responsible until the receiver acknowledges understanding and acceptance.

For application tracking, distinguish requested, received, accepted, and resolved. Assign local deadlines and overdue handling, including after-hours coverage. Workflow Automation develops this coordination contract.

A patient-facing voice demonstration makes the boundary concrete: after additional symptom information, the assistant recommends immediate nurse involvement. The recording demonstrates a request, not completion of a transfer. Likewise, a code-controlled routing layer can ensure that a selected clinical path runs before ordinary conversational generation, but deterministic execution does not establish correct recognition of urgency. Both detection and accepted takeover need assessment.

Commitment at the point of care

A workflow trigger starts consideration of work: a clinician request, incoming result, or scheduled review. Point-of-care integration determines where assistance appears, who receives it, and how long it remains relevant. A draft is not finalized documentation; a suggestion is not an order; delivery of an alert is not a completed response.

SEIPS, the Systems Engineering Initiative for Patient Safety model introduced by Pascale Carayon and colleagues in 2006, connects people, tasks, tools, organizational conditions, and physical surroundings to care processes and outcomes. Its work-system perspective explains why a technically correct feature can still create interruptions or transfer workload. Review time and staffing belong in the design. HELP’s earlier data-triggered and time-based support illustrates the enduring importance of delivering information within actual work.

Acceptance must also account for change. A clinician may review a draft before a new result arrives or another user edits the target record. Refresh clinical context when relevant premises change; separately check authority and target-record concurrency. SMART distinguishes selected patient context from granted resource operations. Approval in the application does not create an EHR permission.

FHIR R4’s version-aware updates use ETag and If-Match to reject overwriting a resource whose version changed. Conditional creation can avoid another resource when specified matching criteria identify an existing one. Server support must be checked, and these protections do not prove clinical appropriateness or freshness of every source used by a draft. Authorization at the protected operation and real integration tests cover the underlying mechanics. Test wrong-patient context, intervening changes, rejection, and uncertain submissions as part of the clinical workflow.

Reviewed is not yet committed

Example

Clinical validity, current authority, and target-record concurrency are independent acceptance conditions.

Reviewed proposalProposed content and identified source versions.
Current clinical contextConfirm patient and encounter; reassess relevant source changes.
Current write authorityCheck the requested operation against server permissions.
Target version matchesWhere supported, apply version-aware update protection.
Refresh and reviewNo commitment of this proposal under stale or conflicting conditions.
DeniedDo not perform an unauthorized write.
Commit confirmedAuthoritative result confirms the accepted record change.
A proposed path to a confirmed record change, not completed care. Source freshness is checked separately from the target resource version; a version check cannot certify every clinical premise. Server support is required for version-aware updates. Uncertain submissions go to reconciliation.
Read the diagram as text
  • Reviewed proposal. Proposed content and identified source versions.
  • Current clinical context. Confirm patient and encounter; reassess relevant source changes.
  • Current write authority. Check the requested operation against server permissions.
  • Target version matches. Where supported, apply version-aware update protection.
  • Refresh and review. No commitment of this proposal under stale or conflicting conditions.
  • Denied. Do not perform an unauthorized write.
  • Commit confirmed. Authoritative result confirms the accepted record change.
  • Reviewed proposal → Current clinical context: Submission requested.
  • Current clinical context → Refresh and review: Context invalid or materially changed.
  • Current clinical context → Current write authority: Context remains applicable.
  • Current write authority → Denied: Permission absent.
  • Current write authority → Target version matches: Permission granted.
  • Target version matches → Refresh and review: Version conflict.
  • Target version matches → Commit confirmed: Match and successful update.

VI. Evidence for use

Validate the task and population

Clinical validation assesses clinically relevant performance for the intended task, population, and care context. IMDRF’s clinical-evaluation framework distinguishes a valid clinical association between output and condition, reliable production of the intended output from inputs, and achievement of the intended clinical purpose. Technical correctness, clinical performance, workflow behavior, and patient outcomes remain separate claims; Evals and Benchmarks develops that general distinction.

A reference standard is the procedure used to establish the comparison answer. It may involve independent expert assessment, adjudication of disagreement, or later outcome ascertainment. Agreement with a comparator that does not establish condition status cannot automatically be called diagnostic sensitivity or specificity. Report counts and sampling uncertainty, not percentages alone. More observations reduce sampling uncertainty but do not remove a biased reference procedure.

The distinction predates modern models. Yu and colleagues’ 1979 MYCIN evaluation asked eight outside specialists to assess anonymized prescriptions for ten deliberately challenging retrospective meningitis cases. The study assessed expert acceptability, with disagreement among evaluators. It did not establish routine-care outcomes, cost effectiveness, or deployment readiness.

The target itself can be wrong. Obermeyer and colleagues’ 2019 study examined a system using predicted healthcare spending as a proxy for need. Black patients were sicker than White patients at the same score, with lower spending at comparable illness burden reflecting unequal access and utilization. Accurate cost prediction could therefore allocate help poorly. The paper illustrates target mismatch, not merely a model fitting error.

A suitable target can still fail to transfer. A 2021 external validation of a widely deployed sepsis predictor found poor performance in one academic center and examined calibration, alert burden, and added value relative to existing practice. This was a historical evaluation of a specified model and cohort, not a judgment about every version or hospital. Language, age, comorbidity, equipment, and access patterns can all define materially different use conditions.

Match separation to the intended claim, retaining only information available when the system would run. The detailed methods belong in Data Quality and Curation.
Evaluation boundaryMain transfer questionRemaining limitation
Patient separationPerformance on unseen people.Does not establish future or cross-site performance.
Later periodPerformance as time and practice change.Returning patients may still overlap.
Another institutionPerformance under another site's conditions.One external site does not represent every setting.

Errors, probabilities, and workload

Sensitivity is the fraction of actual cases detected. Specificity is the fraction of non-cases correctly left unflagged. Positive predictive value is the fraction of alerts that are actual cases. Prevalence is the proportion of the evaluated population with the target condition or event. Sensitivity therefore does not tell a reviewer how often an alert is correct.

Hold sensitivity and specificity fixed to isolate prevalence. Real population changes can also alter those rates.

Prevalence changes the alert workload

Hypothetical cohort of 1,000; sensitivity 80% and specificity 90% stay fixed. Change only prevalence to isolate its effect on alerts.

170 alerts: 80 actual cases + 90 false alerts.
Positive predictive value: 80/170 = 47.1%. 20 actual cases are missed.
Alert counts with fixed sensitivity and specificity80 detected cases and 90 false alerts out of 170 alerts. Same 0 to 250 alert scale at every prevalence.050100150200250Detected cases: 80False alerts: 90All alerts: 170 · counts, not review minutes or treatment benefit
Actual casesDetectedMissedFalse alertsCorrectly unflaggedCohort
1008020908101,000
Reference comparison · available without changing controls
PrevalenceDetectedMissedFalse alertsAll alertsPPV
1%82991077.5%
10%80209017047.1%
The calculation holds sensitivity and specificity fixed, not a clinical threshold or a treatment policy. Lower prevalence changes both alert composition and volume. Actual review time, response delays and clinical benefit must be measured separately.

Lower prevalence leaves fewer real cases while false alerts still arise from the much larger non-case population. Review demand must therefore be assessed alongside detection: count alerts and missed cases, then measure the actual work and response delays they create. Changing an action threshold changes that workload and its errors; available capacity is a constraint, not evidence that an arbitrary higher threshold is clinically acceptable.

Calibration concerns absolute probabilities. Among comparable patients assigned approximately 10% risk for a defined event and horizon, approximately 10% should experience it. Ranking higher-risk patients well does not establish calibrated probabilities, and calibration at one institution may not transfer. Nor does calibration select an appropriate intervention. See Interpret probability forecasts. A confidence interval for measured sensitivity answers another question: uncertainty in a population performance estimate, not an individual patient’s event probability.

For notes and conversations, measure the errors that matter to that task rather than forcing every output into a diagnostic score. Clinicians can identify unsupported additions, omissions, incorrect intervention categories, or intervention at the wrong conversational turn. In Evals Driven-Development for a Mental Health AI Coach, clinician annotations become typed regression cases with conversation input, an expected observation, category, and replay turn. Both missed and inappropriate interventions matter. If a model grades these cases, validate the judge against independent clinical assessment; a generated critique is not its own reference standard.

Evaluate the complete intervention

Retrospective evaluation uses previously collected cases. Prospective evaluation is planned around newly occurring work. Silent operation runs on current inputs while withholding outputs from care decisions. It can reveal input and performance problems, but cannot show how clinicians respond to advice they never see. Usual care is the actual existing alternative, including its staffing, tools, and follow-up—not simply the absence of a model.

Choose the design for the remaining claim; these are not mandatory steps in a universal approval ladder.
DesignExposureWhat it can investigate
Retrospective testingPreviously collected cases.Specified output performance under reconstructed conditions.
Silent prospective operationNew inputs; outputs withheld from care.Current data availability and prospective performance.
Supervised live evaluationUsers see and may act on outputs.Reliance, usability, workflow changes, and observed failures.
Comparative outcome studyDefined assisted and comparison workflows.Differences in prespecified outcomes under the study design.

DECIDE-AI, published in 2022, provides reporting guidance for early live evaluation, including actual use, overrides, workflow, and harm. CONSORT-AI, published in 2020, extends clinical-trial reporting to describe algorithm version, inputs, users, outputs, and effects on decisions. Both make the human–AI intervention more inspectable; neither proves effectiveness or supplies authorization to deploy.

Random assignment uses an unpredictable chance process rather than prognosis or preference to allocate an intervention. Concealing the next assignment prevents recruitment from being steered toward a particular arm. Assignment may be by patient, clinician, or care unit: shared staff and practices can let one arm affect another, so the unit must fit the workflow. Concurrent observation alone does not randomize exposure. Choose the live experiment develops these design choices. Clinical randomized studies are not categorically prohibited; exposure and safeguards require an appropriate study and institutional review.

Keep endpoints separate. The ambient-documentation trial measured practitioner and documentation outcomes; ACCESS distinguished screening from indicated follow-up. Neither established the patient-health claims those intermediate outcomes might suggest. A surrogate endpoint substitutes another measure for an outcome directly relevant to patients. FDA–NIH BEST emphasizes that correlation with health outcomes is insufficient: intervention-induced changes in the surrogate must reliably predict benefit in the specified context. An efficiency gain, a clinical benefit, a safety assessment, and permission to operate therefore require different support.

VII. Operation and recovery

Monitor changing conditions

Distribution shift means operating conditions differ from those evaluated. Patient mix, measurement practices, interfaces, protocols, staffing, and model versions can change different parts of the system. A detected change warrants investigation; it does not alone establish degraded clinical performance. Machine Learning Fundamentals explains the broader concept. Monitoring should preserve separate views of input availability, meaning, output quality, response workload, and outcomes.

Assign clinical and technical owners according to the changed relationship.
Observed changeInitial investigationWhat remains unproven
Fewer results arriveInspect source and interface availability.Whether affected patients have normal findings.
A field or unit changesReview mapping with source-system and clinical owners.Whether prior interpretation remains valid.
More alerts remain pendingInspect case mix, review effort, staffing, and acceptance.Whether detector performance itself deteriorated.
Recent outcomes are unavailableTrack follow-up maturity and ascertainment.Clinical performance in that incomplete cohort.

Missing documentation, delayed outcomes, and selective follow-up are different problems. An absent historical entry might never have been recorded. A recent outcome may not yet be observable. Follow-up confined to selected patients can leave the others without comparable labels. Do not classify these unknowns as successes or negative outcomes; incomplete feedback requires explicit accounting.

Actions can also change the outcome being monitored. In the clinical AI quality-improvement framework’s hypothetical early-warning example, an alert prompts treatment and the predicted event does not occur. The record alone cannot distinguish a false prediction from successful prevention: the outcome without treatment is unobserved. Simple prediction-versus-outcome scoring can therefore misdescribe the intervention’s effect.

A non-event leaves the treatment effect unresolved

Same patient, one observed course. The alternative is not a second patient or a measured study arm.

Observed course

  1. Alert prompts treatment.
  2. Treatment is followed by no observed event.

Counterfactual · unobserved

Without that treatment, what would have happened to this same patient?

Outcome unknown
In this hypothetical example, no event is observed after treatment. That sequence alone cannot distinguish an incorrect prediction from successful prevention: the same patient’s outcome without treatment is unobserved.

Retain source and model versions, relevant policy and workflow versions, observed effects, and reviewer decisions under defined access and retention controls. Sensitive telemetry needs its own governance. Clinical and technical owners should agree on conditions for reassessment, restricted use, and suspension. A model or prompt change requires renewed evaluation of affected behavior; a healthy request pipeline is not a substitute.

Recovery and affected work

Recovery begins by identifying what failed. Degraded operation deliberately provides a smaller supported service. Reconciliation determines what actually happened and resolves discrepancies against authoritative records. A record amendment corrects or qualifies documentation while preserving attributable history. These responses solve different problems; restoring a model version addresses future execution, not every effect already produced.

FailureRequired response
Assistance unavailableContinue through an established non-AI workflow; retain ownership of pending work.
Required source records unavailableWithhold dependent conclusions or clearly bound any partial information.
Submission outcome uncertainInspect authoritative status and use supported duplicate-protection mechanisms before repetition.
Accepted output known to be wrongReview affected work, correct through the applicable record process, and assign necessary communication.

The SAFER contingency-planning guide recommends tested downtime procedures, named clinical and technical leadership, communication independent of failed infrastructure, and alternative documentation. Restored systems must reconcile information captured during downtime. Suspending assistance must not silently abandon urgent, overdue, or unaccepted work.

An uncertain write is not a confirmed failure, and resource-level duplicate protection is not proof that a clinical action occurred exactly once. A known wrong note has another remedy: correction with history. The historical CMS amendment guidance illustrates preserving distinguishable original and changed content with authorship and dates. It is not a determination of current local policy. Nor can a record correction undo an utterance already heard or an action already taken.

Restoring software without erasing effects requires separate completion evidence. Before resuming the affected use, verify repaired dependencies and an evaluated operating mode, reconcile pending effects, and confirm ownership of unfinished work. Long-running corrective work may continue under an accountable plan. The final measure of recovery is a workable care service with understood obligations—not merely a responding endpoint.

Open questions

  1. Detection can expand faster than follow-up capacity. Establishing useful expansion requires studying completed indicated care, delays, and workload across services—not screening volume alone. Progress would demonstrate that additional detection reaches appropriate follow-up without leaving less visible groups behind.

  2. Effective oversight must remain effective under routine workload. Reviewers can accept incorrect advice and reject useful advice, while repeated alerts change response behavior. Progress would show sustained error detection and correction in real workflows, with review effort and downstream consequences measured together.

  3. Clinical monitoring becomes difficult when outcomes arrive late or treatment changes them. A non-event after an alert may reflect successful intervention rather than a false alarm. Progress requires evaluation designs that separate prediction quality, response behavior, and intervention effects while keeping unavailable outcomes explicit.

  4. Local adaptation must accommodate specialty and organizational differences without silently changing clinical authority. Decentralized expert review can improve contextual fit, but safe update methods and cross-setting regression evidence remain necessary. Progress would make each local change attributable, reviewable, and independently assessable.

Follow the curated reading path through the speakers and demonstrations behind this entry.

19 min

AI Engineer World's Fair 2026 · 2026

Shipping AI to a Million Patients Without an A/B Test

Jared Joselowitz

Cited in this entry

Explains hazard-driven testing for patient conversations and why rollback cannot retract delivered speech. Read its deployment argument as motivation for pre-exposure evidence, not a blanket prohibition on clinical randomized studies.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

48 matching talks

TalkSpeakerEventYear
Vivek MuppallaAI Engineer World's Fair 20262026
Rashi AgrawalAI Engineer World's Fair 20262026
Vasant KearneyAI Engineer World's Fair 20262026
Frank CoyleAI Engineer World's Fair 20262026
Hamed Firooz, Maziar SanjabiAI Engineer World's Fair 20252025
Yu SuAI Engineer World's Fair 20262026
Apoorva JoshiAI Engineer World's Fair 20262026
Roy DerksAI Engineer Summit 20252025
Adam TerlsonAI Engineer Summit 20252025
How to Build Trustworthy AI

Transcript reviewed

Allie HoweAI Engineer World's Fair 20252025
Angel Ortmann LeeAI Engineer World's Fair 20262026
Ian ButlerAI Engineer World's Fair 20252025
Jun Yu TanAI Engineer World's Fair 20252025
Jesse HuAI Engineer Code 20252025
Angus J. McLeanAI Engineer Europe 20262026
Christopher LovejoyAI Engineer Europe 20262026
AI’s Jurassic Park Period

Transcript reviewed

Aaron StanleyAI Engineer World's Fair 20262026
Jeremy Silva, Chris HernandezAI Engineer World's Fair 20252025
Mohak SharmaAI Engineer Summit 20252025
Anna Marie BenzonAI Engineer World's Fair 20262026
Giran Moodley, Mayan Soni, Oussama Hafferssas, Mayank SoniAI Engineer Europe 20262026
Sandipan BhaumikAI Engineer Europe 20262026
Ayush BhardwajAI Engineer World's Fair 20262026
Philip RathleAI Engineer World's Fair 20242024
Christopher Lovejoy, Saul HowardAI Engineer World's Fair 20262026
Dan MasonAI Engineer World's Fair 20252025
Sandipan BhaumikAI Engineer Europe 20262026
Agents Need Feature Flags

Transcript reviewed

Sachin GuptaAI Engineer World's Fair 20262026
Dan FengAI Engineer World's Fair 20262026
Clay Cockrell, Tony FabrikantAI Engineer World's Fair 20262026
Denys LinkovAI Engineer World's Fair 20262026
Anant ShankhdharAI Engineer World's Fair 20262026
Stephen ChinAI Engineer Europe 20262026
Andreas Kollegger, Zaid ZaimAI Engineer Europe 20262026
Don't be data poor

Metadata candidate

Anuj IravaneAI Engineer World's Fair 20262026
Rossella Blatt Vital, Deepsha MenghaniAI Engineer World's Fair 20252025
Mike BursellAI Engineer World's Fair 20252025
Kwindla Hultman KramerAI Engineer World's Fair 20242024
Christopher LovejoyAI Engineer World's Fair 20252025
Kwindla Kramer, Shrestha Basu MallickAI Engineer World's Fair 20252025
Nik CaryotakisAI Engineer Summit 20252025
Akram BaharloueiAI Engineer World's Fair 20262026
Daniel WhitenackAI Engineer World's Fair 20242024
Denys LinkovEvent unknownUnknown
Anant SrivastavaAI Engineer Code 2025Unknown
Sebastian FoxAI Engineer World's Fair 20262026
William LyonEvent unknownUnknown
Derek MeeganAI Engineer Code 2025Unknown

References

Coverage and source review
Processed transcripts
34 processed in full · 6 in the curated path
Automated source review
Passed
Metadata candidates
20 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. AHRQ: What is workflow?

    https://digital.ahrq.gov/health-it-tools-and-resources/evaluation-resources/workflow-assessment-health-it-toolkit/workflow

  2. IHI: Closing the Loop—A Guide to Safer Ambulatory Referrals in the EHR Era

    https://www.ihi.org/library/publications/closing-loop-guide-safer-ambulatory-referrals-ehr-era

  3. TRIPOD+AI expanded checklist: timing, labels, and evaluation separation

    https://www.tripod-statement.org/wp-content/uploads/2024/04/TRIPODAI-Supplement.pdf

  4. Use of an Electronic Referral System to Improve the Outpatient Primary Care–Specialty Care Interface

    https://digital.ahrq.gov/sites/default/files/docs/citation/ereferralimplementationreportfinal.pdf

  5. Kaiser Permanente: Early Warning System for Hospitalized Patients

    https://www.kpihp.org/blog/integrated-care-stories-early-warning-system-for-hospitalized-patients/

  6. WHO: Interagency Integrated Triage Tool

    https://www.who.int/tools/triage

  7. Accuracy and safety of an autonomous artificial intelligence clinical assistant conducting telemedicine follow-up assessment for cataract surgery

    https://iaensalud.es/wp-content/uploads/2024/10/36.-2024.-Accuracy-and-safety-of-an-autonomous-artificial-intelligence-clinical-assistant-conducting-telemedicine-follow-up-assessment-for-cataract-surgery.pdf

  8. How to Leverage Domain Expertise — Chris Lovejoy, Notius Labs

    https://www.youtube.com/watch?v=kfSDc2eVLo4&t=1032

  9. eCQI Resource Center: Clinical Decision Support

    https://ecqi.healthit.gov/cds/about

  10. AI System Design: From Idea to Production

    https://www.youtube.com/watch?v=T0HhO4YtTfE&t=109

  11. Google: Supervised learning foundations

    https://developers.google.com/machine-learning/intro-to-ml/supervised

  12. AI System Design: From Idea to Production

    https://www.youtube.com/watch?v=T0HhO4YtTfE&t=901

  13. FDA, Health Canada, and MHRA: Good Machine Learning Practice Guiding Principles

    https://www.fda.gov/media/153486/download

  14. SMART App Launch: Patient context and resource permissions

    https://hl7.org/fhir/smart-app-launch/scopes-and-launch-context.html

  15. IMDRF: SaMD definition and intended medical purpose

    https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf

  16. Ledley and Lusted: Reasoning Foundations of Medical Diagnosis

    https://www.cs.tufts.edu/comp/150AIH/pdf/LedleyLu59.pdf

  17. Pryor, Gardner, Clayton, and Warner: The HELP System

    https://collections.lib.utah.edu/dl_files/c4/66/c46608381eb1ce04f479da8e9940c52425ab1a64.pdf

  18. Buchanan and Shortliffe: Preface to Rule-Based Expert Systems

    https://www.shortliffe.net/Buchanan-Shortliffe-1984/Preface.pdf

  19. Gulshan and colleagues: Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy

    https://jamanetwork.com/journals/jama/fullarticle/2588763

  20. From Ambient Documentation to Clinical Intelligence

    https://www.youtube.com/watch?v=u6q-byPWUuo&t=436

  21. eCQI Resource Center: Electronic health record

    https://ecqi.healthit.gov/glossary/electronic-health-record-ehr

  22. NHS England: Ambient scribing guidance for health and care professionals

    https://digital.nhs.uk/data-and-information/information-governance/guidance/using-ai-enabled-ambient-scribing-products-in-health-and-care-settings/guidance-for-health-and-care-professionals

  23. Abridge: Verify a Note With Linked Evidence

    https://support.abridge.com/hc/en-us/articles/30235128433811-Verify-a-Note-With-Linked-Evidence

  24. A Pragmatic Randomized Controlled Trial of Ambient Artificial Intelligence to Improve Health Practitioner Well-Being

    https://pmc.ncbi.nlm.nih.gov/articles/PMC12858090/

  25. Use of an Electronic Referral System to Improve the Outpatient Primary Care–Specialty Care Interface

    https://digital.ahrq.gov/sites/default/files/docs/citation/ereferralimplementationreportfinal.pdf

  26. AI That Pays: Lessons from Revenue Cycle

    https://www.youtube.com/watch?v=TquUsN1QsWs&t=763

  27. Anterior and Stellarus prior-authorization workflow

    https://www.anterior.com/insights/anterior-partners-with-stellarus-to-allow-health-plans-to-deliver-real-time-ai-driven-prior-authorization

  28. Abràmoff and colleagues: Pivotal Trial of an Autonomous AI-Based Diagnostic System

    https://www.nature.com/articles/s41746-018-0040-6

  29. Wolf and colleagues: The ACCESS Randomized Control Trial

    https://www.nature.com/articles/s41467-023-44676-z.pdf

  30. Hernán and Robins: Counterfactual outcomes and confounding

    https://www.popdata.bc.ca/sites/default/files/documents/events/etu/Hernan_Book.pdf

  31. HL7 FHIR R5: DiagnosticReport

    https://hl7.org/fhir/diagnosticreport.html

  32. HL7 FHIR: MedicationAdministration

    https://hl7.org/fhir/R4B/medicationadministration.html

  33. What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

    https://www.jmir.org/2021/3/e22219

  34. FHIR R5: Observation effective and issued times

    https://hl7.org/fhir/observation-definitions.html

  35. HL7 FHIR: Provenance

    https://hl7.org/fhir/R4/provenance.html

  36. FHIR R5 data-absent-reason value set

    https://hl7.org/fhir/valueset-data-absent-reason.html

  37. HL7 FHIR R5 Overview

    https://hl7.org/fhir/overview.html

  38. FHIR R5 Observation: results, interpretation and absent data

    https://hl7.org/fhir/observation.html

  39. Regenstrief Institute: LOINC Users' Guide—Introduction

    https://loinc.org/kb/users-guide/introduction

  40. HL7 FHIR R5 Datatypes: Coding and CodeableConcept

    https://fhir.hl7.org/fhir/datatypes.html

  41. FHIR R5 Quantity: coded units and comparison semantics

    https://hl7.org/fhir/datatypes.html#Quantity

  42. SMART App Launch: Conformance

    https://www.hl7.org/fhir/smart-app-launch/STU2.1/conformance.html

  43. HL7 FHIR R5: Comparison with HL7 Version 2

    https://fhir.hl7.org/fhir/comparison-v2.html

  44. Why Your Enterprise Tech Stack Isn't Ready for AI Agents - And What to Build Instead

    https://www.youtube.com/watch?v=mav15aW9lLM&t=350

  45. HHS: De-identification methods and residual identification risk

    https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification/index.html

  46. HL7 Clinical Practice Guidelines: CPGPlan

    https://hl7.org/fhir/uv/cpg/documentation-approach-12-03-cpg-plan.html

  47. Mission-Critical Evals at Scale: Learnings from 100,000 Medical Decisions

    https://www.youtube.com/watch?v=cZ5ZJy19KMo&t=60

  48. Citation Needed: Provenance for LLM-Built Knowledge Graphs

    https://www.youtube.com/watch?v=H7puB0RwJMM&t=20

  49. William van Melle: The Structure of the MYCIN System

    https://www.shortliffe.net/Buchanan-Shortliffe-1984/Chapter-04.pdf

  50. Mission-Critical Evals at Scale: Learnings from 100,000 Medical Decisions

    https://www.youtube.com/watch?v=cZ5ZJy19KMo&t=147

  51. FDA, Health Canada, and MHRA: Transparency for Machine Learning-Enabled Medical Devices

    https://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles

  52. AHRQ TeamSTEPPS: Handoff

    https://www.ahrq.gov/teamstepps-program/curriculum/communication/tools/handoff.html

  53. Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists

    https://pmc.ncbi.nlm.nih.gov/articles/PMC12008695/

  54. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

    https://link.springer.com/article/10.1186/s12911-017-0430-8

  55. NIST AI RMF 1.0: accountability, appeals and override

    https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf

  56. 200 Million Patient Interactions Later: What the Generic Voice Stack Misses

    https://www.youtube.com/watch?v=AN65uc645mE&t=263

  57. Guardrails First: Engineering Member-Facing Health AI

    https://www.youtube.com/watch?v=YXEqC05WEI0&t=379

  58. Carayon and colleagues: Work System Design for Patient Safety—The SEIPS Model

    https://www.researchgate.net/publication/6655634_Work_system_design_for_patient_safety_the_SEIPS_model

  59. HL7 FHIR HTTP: Version-aware updates and conditional creation

    https://hl7.org/fhir/R4/http.html

  60. IMDRF: Software as a Medical Device—Clinical Evaluation

    https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-170921-samd-n41-clinical-evaluation_1.pdf

  61. FDA: Sampling uncertainty in diagnostic test performance

    https://www.fda.gov/regulatory-information/search-fda-guidance-documents/statistical-guidance-reporting-results-studies-evaluating-diagnostic-tests-guidance-industry-and-fda

  62. Yu and colleagues: An Evaluation of MYCIN's Advice

    https://www.shortliffe.net/Buchanan-Shortliffe-1984/Chapter-31.pdf

  63. Obermeyer et al.: Healthcare cost as a biased proxy for need

    https://www.ftc.gov/system/files/documents/public_events/1548288/privacycon-2020-ziad_obermeyer.pdf

  64. External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients

    https://pmc.ncbi.nlm.nih.gov/articles/PMC8218233/

  65. Classification: Accuracy, Recall, Precision and Related Metrics

    https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall

  66. Calibration: the Achilles Heel of Predictive Analytics

    https://link.springer.com/article/10.1186/s12916-019-1466-7

  67. NIST: What a confidence level means

    https://www.itl.nist.gov/div898/handbook/prc/section1/prc14.htm

  68. Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & Safely

    https://www.youtube.com/watch?v=O72p-rBb2bA&t=712

  69. Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & Safely

    https://www.youtube.com/watch?v=O72p-rBb2bA&t=847

  70. From Ambient Documentation to Clinical Intelligence

    https://www.youtube.com/watch?v=u6q-byPWUuo&t=825

  71. DECIDE-AI: Early clinical workflow, safety, and human-factors evidence

    https://www.nature.com/articles/s41591-022-01772-9/tables/2

  72. CONSORT pragmatic-trial extension: Applicability to usual care

    https://dfcm.utoronto.ca/sites/default/files/inline-files/zwarenstein_consort_pragmatictrialextension.pdf

  73. CONSORT-AI: reporting clinical trials of AI interventions

    https://www.nature.com/articles/s41591-020-1034-x

  74. CONSORT 2010 explanation: random assignment and concealment

    https://www.limbic-cenc.org/wp-content/uploads/CONSORT-2010-Explanation-and-Extension-JCE10.pdf

  75. FDA–NIH BEST: Validated surrogate endpoints

    https://www.ncbi.nlm.nih.gov/books/NBK453484/

  76. Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare

    https://www.nature.com/articles/s41746-022-00611-y.pdf

  77. NIST AI RMF: Monitoring, incident response, and recovery

    https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf

  78. SAFER Guide: Contingency Planning

    https://healthit.gov/wp-content/uploads/2025/06/SAFER-Guide-2.-Contingency-Planning-Final.pdf

  79. CMS Transmittal 442: Amendments, Corrections and Delayed Entries in Medical Documentation

    https://www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/Downloads/R442PI.pdf

  80. Shipping AI to a Million Patients Without an A/B Test

    https://www.youtube.com/watch?v=McknwOzbmyg&t=66

  81. NIST SP 800-61r3: incident response and verified recovery

    https://doi.org/10.6028/NIST.SP.800-61r3

  82. Understanding AI Stakes to Break Production Code

    https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1189

  83. Biases in Electronic Health Record Data Due to Processes Within the Healthcare System

    https://www.bmj.com/content/361/bmj.k1479

  84. Why Your Enterprise Tech Stack Isn't Ready for AI Agents - And What to Build Instead

    https://www.youtube.com/watch?v=mav15aW9lLM&t=425

  85. The Target Trial Framework for Causal Inference From Observational Data: Why and When Is It Helpful?

    https://pubmed.ncbi.nlm.nih.gov/39961105/

  86. Demystifying evals for AI agents

    https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

  87. Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCP

    https://www.youtube.com/watch?v=sn79oS4MZFI&t=911

  88. How to Leverage Domain Expertise — Chris Lovejoy, Notius Labs

    https://www.youtube.com/watch?v=kfSDc2eVLo4&t=780

  89. How to Leverage Domain Expertise — Chris Lovejoy, Notius Labs

    https://www.youtube.com/watch?v=kfSDc2eVLo4&t=932

  90. Citation Needed: Provenance for LLM-Built Knowledge Graphs

    https://www.youtube.com/watch?v=H7puB0RwJMM&t=316

  91. Mission-Critical Evals at Scale: Learnings from 100,000 Medical Decisions

    https://www.youtube.com/watch?v=cZ5ZJy19KMo&t=223

  92. Shipping AI to a Million Patients Without an A/B Test

    https://www.youtube.com/watch?v=McknwOzbmyg&t=233