AI Engineer World's Fair 2025

Critical AI Inference Your CIO Can Trust

Loading slides…

Read the talk

Critical AI Inference Your CIO Can Trust

Trustworthy inference requires more than accurate predictions: teams need explanations, traceable decisions, controls that can intervene, and a way to connect failures to business costs.

From a talk by Sahil Yadav and Hariharan Ganesan

Can we trust the inference?

Can we trust an AI inference when it changes a business decision—or determines whether someone is safe? Across health monitoring, industrial IoT and telecom automation, Sahil Yadav and Hariharan Ganesan have repeatedly encountered that question. In these environments, trust concerns the consequences of a particular decision, not just whether a model performs well in aggregate.

Yadav opens with three figures: McKinsey’s 78% reporting organizational AI use, EY’s 95% reporting AI investment, and an 11% figure he describes as governance focus. The McKinsey measure includes use in at least one business function, including experimentation; EY’s investment survey does not imply mature deployment. The governance comparison needs particular care: the matching figure in PwC’s responsible-AI research concerns fully implemented fundamental capabilities, a narrower condition than simply focusing on governance. Yadav’s 67-percentage-point gap is therefore a comparison across measures, not a measured gap within one survey population.

Slide titled “The Trust Gap in Enterprise AI,” with three columns covering rapid adoption, lack of mature governance practices, and a significant trust gap, above a highlighted takeaway.Open full source frame
The trust gap between enterprise AI adoption and mature governance.

The operational examples make the concern concrete: an AI decision disrupts a telecom network, a gas sensor’s data is misinterpreted and puts people at risk, and supply-chain AI confuses SKUs and produces losses. Yadav invokes AT&T and Verizon to convey the scale of network downtime, not to identify either as the operator in the example. These are silent failures: consequences can accumulate before a team understands what went wrong or how much damage it has caused.

0:280:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:28 · section reference included

Explain, reconstruct, intervene

Yadav separates three requirements that are easy to conflate:

RequirementOperational purpose
ExplainabilityUnderstand the basis for an inference.
TraceabilityReconstruct the data and changes behind it.
GuardrailsStop the system at a defined boundary.

An explanation helps someone understand a result. An audit trail acts like a flight recorder, allowing investigators to recreate the situation. A guardrail limits what the system may do next. All three must remain usable as deployment scales.

Ganesan develops this into an operational formulation, starting from the safety expectations already applied to aircraft, energy grids and banking. An important AI decision should come with a plain-English explanation that a user, decision maker or auditor can act on without needing a data scientist to translate it. Actionable intelligibility is the requirement: the explanation must help someone make the next decision.

Next comes adaptive control. If a system begins making risky decisions, it should be able to slow down, change course or ask for help—the equivalent of lane assistance. Human involvement then needs rules and playbooks: notify the right expert, at the right time, with enough information to act, without overwhelming either the person or the system.

In this second formulation, traceability is the foundation beneath explanations, adaptive controls and human involvement. Ganesan describes digitally signed, trackable data and changes, drawing comparisons to a software bill of materials and package tracking. The purpose is continuity: from the origin of an input through its transformations, the team should be able to follow what happened.

2:573:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:57 · section reference included

Put trust into the AI lifecycle

The speakers call their integrated approach XTOps; the lifecycle slide labels it XDOps. Its purpose is to extend MLOps with explicit trust controls and human oversight throughout the lifecycle, rather than attach them only after a model is running.

The progression follows the work itself:

  1. Data: record where inputs originate and how they change.
  2. Training: pursue actionable intelligibility alongside accuracy, so changes in the basis for decisions can become visible.
  3. Deployment: apply adaptive guardrails that respond to new data or situations and pause when drift needs investigation.
  4. Operation and feedback: let people intervene and feed real-world observations back into improvements.

Each stage supplies something the next stage needs: provenance, an understandable decision basis, intervention capability, and feedback.

Diagram titled “From MLOps to XDOps: Closing the Trust Gap,” showing Data, Train, Deploy, and Monitor connected beneath traceability, intelligibility, adaptive control, and human–AI teaming.Open full source frame
XDOps adds traceability, intelligibility, adaptive control, and human–AI teaming across the AI lifecycle.

The resulting decision should have a why, a when and a who attached to it. That makes launching a model only part of the job; the operating system around it must also make its behavior understandable and accountable.

6:106:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:10 · section reference included

Upgrade the controls already in MLOps

Many of these components already exist in MLOps. Ganesan presents adoption as an integration effort, not a switch or a wholesale replacement. The proposed difference lies in how existing components support trust:

Existing capabilityProposed addition
IAM and security policiesContext-sensitive controls on risky AI decisions
Operational metricsTrust dashboards understandable to leadership
Human feedbackStructured review and correction workflows

IAM can establish whether an actor has permission to perform an operation. An AI-aware guardrail must also evaluate whether a particular decision is risky in its current context. Likewise, a dashboard needs to communicate trust concerns beyond the engineering team, and a human feedback path needs to make correction routine rather than ad hoc.

A small TypeScript decision gate illustrates the control boundary. Policy evaluation produces a disposition before anything is executed; a risky proposal becomes a review item rather than an action:

typescript

type Proposal = {
  id: string;
  explanation: string;
};

type PolicyResult =
  | { allow: true }
  | { allow: false; reason: string; reviewer: string };

function routeProposal(
  proposal: Proposal,
  policy: PolicyResult
) {
  if (!policy.allow) {
    return {
      status: "pending-review" as const,
      proposal,
      reason: policy.reason,
      reviewer: policy.reviewer
    };
  }

  return { status: "permitted" as const, proposal };
}

The function preserves the proposal and its explanation while making the next step explicit. A permitted result still needs an execution path; a pending result needs a review workflow. The broader goal is to spend less engineering time firefighting unpredictable behavior and more time improving the product.

7:568:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:56 · section reference included

Measure diagnosis and business exposure

Ganesan proposes two measures. MTTRE, or mean time to resolve explainable errors, concerns the time needed to understand an unexpected result and respond with a fix. Trust-adjusted risk in dollars concerns the business cost when trust breaks: fines, lost customers and reputational damage. The presentation gives these measures operational meanings, but no formal calculation method.

The connection is duration. Ganesan says some of the cases he discusses take several months to resolve. A system making biased decisions throughout that interval can keep adding harm while the team investigates. The cost extends beyond a fine to engineering effort, regulatory scrutiny and loss of brand value.

Ganesan cites potential incident costs up to $700 million for examples such as privacy bugs or credit-system bias, without supplying a case breakdown. The useful operational question is therefore not simply whether a model failed, but how quickly the organization can understand and contain a failure before its consequences grow.

9:439:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:43 · section reference included

Guardhat: when false alerts teach workers to ignore warnings

At Guardhat, Yadav’s former employer, the application was worker safety in hazardous environments. Wearable IoT devices collected health and environmental data, then sent it to a backend for real-time analysis. The intended chain was straightforward: detect the conditions preceding an incident, warn in time, and help prevent it. GPS was one of the platform’s inputs.

Yadav reports that 70% of cases were false positives in the GPS-related failure, a figure recognized retrospectively rather than during the initial investigation. Repeated false alerts changed worker behavior: people stopped responding. The failure therefore crossed from inference quality into the effectiveness of the safety system itself, creating both danger for workers and liability for the company.

In Yadav’s conventional-resolution breakdown, identifying the problem consumes 70% of the time and finding a solution another 20%, followed by deployment. The central obstacle was the absence of a system to identify GPS drift. With complicated models and code, tracing the alerts back to the faulty input was difficult.

The proposed XTOps response makes that missing diagnostic capability explicit:

Point in the illustrative responseEvent
Day 0An incident includes an ignored alert.
Day 2Attribution telemetry flags the anomaly.
Day 7A GPS-drift fix or workaround is deployed.

Attribution telemetry is the bridge between observing a bad outcome and locating the input responsible for it. The proposed remedy can address the drift or route around it; the talk does not specify the alternative route.

The original incident took eight months to resolve, not seven days. Yadav says that experience helped create the framework; he subsequently reports that comparable problems could be solved in seven days, with enterprise testing leading to adoption as an enterprise standard. The timeline above illustrates the framework’s intended response, not the history of the original repair.

12:1812:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:18 · section reference included

Make the financial case—and restore the alert workflow

The CIO-facing question is what this work saves. Yadav estimates approximately $2.5 million in risk exposure per site per year and reports $500,000 in annual fine savings per site from the product or framework. These are his case-study figures, rather than a general return-on-investment model.

There is also an indirect benefit: a malfunctioning safety system delivers only part of its intended incident-prevention value. Restoring its operation can recover more of that value. Yadav does not quantify the prevention percentage, and the account does not establish that the system eliminated all incidents.

The reported operational outcomes are fewer false alerts, a higher trust score and workers responding to alerts again. More concretely, telemetry helps explain why an inference occurred, controls permit switching when GPS drift appears, and a dashboard notifies a person who can act and retrain the model. The trust score is not defined numerically; the visible behavioral result is that alerts became useful to workers again.

Safety-critical AI case study slide with Challenge and Trust Gaps panels, a Day 0–Day 7 mitigation timeline, and Outcome Highlights and XDOps Contributions panels.Open full source frame
Guardhat case study: GPS drift, ignored alerts, and the mitigation timeline.
13:3615:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:44 · section reference included

What does reputational damage cost?

The closing audience question tests the financial framing: how should an organization measure reputational damage within trust-adjusted risk? Has that damage actually been measured or investigated?

The answer leaves a real limit in place. Reputational harm belongs to the same class of silent failures whose impact is difficult to quantify and predict. Teams may attempt to assign dollar values, but the speakers offer no reliable valuation method. Trust-adjusted risk can make business consequences part of the engineering discussion without making every consequence precisely measurable.

18:0518:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:05 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…