AI Engineer World's Fair 2025
Critical AI Inference Your CIO Can Trust
Loading slides…
Read the talk
Critical AI Inference Your CIO Can Trust
Trustworthy inference requires more than accurate predictions: teams need explanations, traceable decisions, controls that can intervene, and a way to connect failures to business costs.
From a talk by Sahil Yadav and Hariharan Ganesan
Can we trust the inference?
Can we trust an AI inference when it changes a business decision—or determines whether someone is safe? Across health monitoring, industrial IoT and telecom automation, Sahil Yadav and Hariharan Ganesan have repeatedly encountered that question. In these environments, trust concerns the consequences of a particular decision, not just whether a model performs well in aggregate.
Yadav opens with three figures: McKinsey’s 78% reporting organizational AI use, EY’s 95% reporting AI investment, and an 11% figure he describes as governance focus. The McKinsey measure includes use in at least one business function, including experimentation; EY’s investment survey does not imply mature deployment. The governance comparison needs particular care: the matching figure in PwC’s responsible-AI research concerns fully implemented fundamental capabilities, a narrower condition than simply focusing on governance. Yadav’s 67-percentage-point gap is therefore a comparison across measures, not a measured gap within one survey population.
The operational examples make the concern concrete: an AI decision disrupts a telecom network, a gas sensor’s data is misinterpreted and puts people at risk, and supply-chain AI confuses SKUs and produces losses. Yadav invokes AT&T and Verizon to convey the scale of network downtime, not to identify either as the operator in the example. These are silent failures: consequences can accumulate before a team understands what went wrong or how much damage it has caused.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Explain, reconstruct, intervene
Yadav separates three requirements that are easy to conflate:
| Requirement | Operational purpose |
|---|---|
| Explainability | Understand the basis for an inference. |
| Traceability | Reconstruct the data and changes behind it. |
| Guardrails | Stop the system at a defined boundary. |
An explanation helps someone understand a result. An audit trail acts like a flight recorder, allowing investigators to recreate the situation. A guardrail limits what the system may do next. All three must remain usable as deployment scales.
Ganesan develops this into an operational formulation, starting from the safety expectations already applied to aircraft, energy grids and banking. An important AI decision should come with a plain-English explanation that a user, decision maker or auditor can act on without needing a data scientist to translate it. Actionable intelligibility is the requirement: the explanation must help someone make the next decision.
Next comes adaptive control. If a system begins making risky decisions, it should be able to slow down, change course or ask for help—the equivalent of lane assistance. Human involvement then needs rules and playbooks: notify the right expert, at the right time, with enough information to act, without overwhelming either the person or the system.
In this second formulation, traceability is the foundation beneath explanations, adaptive controls and human involvement. Ganesan describes digitally signed, trackable data and changes, drawing comparisons to a software bill of materials and package tracking. The purpose is continuity: from the origin of an input through its transformations, the team should be able to follow what happened.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put trust into the AI lifecycle
The speakers call their integrated approach XTOps; the lifecycle slide labels it XDOps. Its purpose is to extend MLOps with explicit trust controls and human oversight throughout the lifecycle, rather than attach them only after a model is running.
The progression follows the work itself:
- Data: record where inputs originate and how they change.
- Training: pursue actionable intelligibility alongside accuracy, so changes in the basis for decisions can become visible.
- Deployment: apply adaptive guardrails that respond to new data or situations and pause when drift needs investigation.
- Operation and feedback: let people intervene and feed real-world observations back into improvements.
Each stage supplies something the next stage needs: provenance, an understandable decision basis, intervention capability, and feedback.
The resulting decision should have a why, a when and a who attached to it. That makes launching a model only part of the job; the operating system around it must also make its behavior understandable and accountable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Upgrade the controls already in MLOps
Many of these components already exist in MLOps. Ganesan presents adoption as an integration effort, not a switch or a wholesale replacement. The proposed difference lies in how existing components support trust:
| Existing capability | Proposed addition |
|---|---|
| IAM and security policies | Context-sensitive controls on risky AI decisions |
| Operational metrics | Trust dashboards understandable to leadership |
| Human feedback | Structured review and correction workflows |
IAM can establish whether an actor has permission to perform an operation. An AI-aware guardrail must also evaluate whether a particular decision is risky in its current context. Likewise, a dashboard needs to communicate trust concerns beyond the engineering team, and a human feedback path needs to make correction routine rather than ad hoc.
A small TypeScript decision gate illustrates the control boundary. Policy evaluation produces a disposition before anything is executed; a risky proposal becomes a review item rather than an action:
typescript
type Proposal = {
id: string;
explanation: string;
};
type PolicyResult =
| { allow: true }
| { allow: false; reason: string; reviewer: string };
function routeProposal(
proposal: Proposal,
policy: PolicyResult
) {
if (!policy.allow) {
return {
status: "pending-review" as const,
proposal,
reason: policy.reason,
reviewer: policy.reviewer
};
}
return { status: "permitted" as const, proposal };
}
The function preserves the proposal and its explanation while making the next step explicit. A permitted result still needs an execution path; a pending result needs a review workflow. The broader goal is to spend less engineering time firefighting unpredictable behavior and more time improving the product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure diagnosis and business exposure
Ganesan proposes two measures. MTTRE, or mean time to resolve explainable errors, concerns the time needed to understand an unexpected result and respond with a fix. Trust-adjusted risk in dollars concerns the business cost when trust breaks: fines, lost customers and reputational damage. The presentation gives these measures operational meanings, but no formal calculation method.
The connection is duration. Ganesan says some of the cases he discusses take several months to resolve. A system making biased decisions throughout that interval can keep adding harm while the team investigates. The cost extends beyond a fine to engineering effort, regulatory scrutiny and loss of brand value.
Ganesan cites potential incident costs up to $700 million for examples such as privacy bugs or credit-system bias, without supplying a case breakdown. The useful operational question is therefore not simply whether a model failed, but how quickly the organization can understand and contain a failure before its consequences grow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Guardhat: when false alerts teach workers to ignore warnings
At Guardhat, Yadav’s former employer, the application was worker safety in hazardous environments. Wearable IoT devices collected health and environmental data, then sent it to a backend for real-time analysis. The intended chain was straightforward: detect the conditions preceding an incident, warn in time, and help prevent it. GPS was one of the platform’s inputs.
Yadav reports that 70% of cases were false positives in the GPS-related failure, a figure recognized retrospectively rather than during the initial investigation. Repeated false alerts changed worker behavior: people stopped responding. The failure therefore crossed from inference quality into the effectiveness of the safety system itself, creating both danger for workers and liability for the company.
In Yadav’s conventional-resolution breakdown, identifying the problem consumes 70% of the time and finding a solution another 20%, followed by deployment. The central obstacle was the absence of a system to identify GPS drift. With complicated models and code, tracing the alerts back to the faulty input was difficult.
The proposed XTOps response makes that missing diagnostic capability explicit:
| Point in the illustrative response | Event |
|---|---|
| Day 0 | An incident includes an ignored alert. |
| Day 2 | Attribution telemetry flags the anomaly. |
| Day 7 | A GPS-drift fix or workaround is deployed. |
Attribution telemetry is the bridge between observing a bad outcome and locating the input responsible for it. The proposed remedy can address the drift or route around it; the talk does not specify the alternative route.
The original incident took eight months to resolve, not seven days. Yadav says that experience helped create the framework; he subsequently reports that comparable problems could be solved in seven days, with enterprise testing leading to adoption as an enterprise standard. The timeline above illustrates the framework’s intended response, not the history of the original repair.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the financial case—and restore the alert workflow
The CIO-facing question is what this work saves. Yadav estimates approximately $2.5 million in risk exposure per site per year and reports $500,000 in annual fine savings per site from the product or framework. These are his case-study figures, rather than a general return-on-investment model.
There is also an indirect benefit: a malfunctioning safety system delivers only part of its intended incident-prevention value. Restoring its operation can recover more of that value. Yadav does not quantify the prevention percentage, and the account does not establish that the system eliminated all incidents.
The reported operational outcomes are fewer false alerts, a higher trust score and workers responding to alerts again. More concretely, telemetry helps explain why an inference occurred, controls permit switching when GPS drift appears, and a dashboard notifies a person who can act and retrain the model. The trust score is not defined numerically; the visible behavioral result is that alerts became useful to workers again.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What does reputational damage cost?
The closing audience question tests the financial framing: how should an organization measure reputational damage within trust-adjusted risk? Has that damage actually been measured or investigated?
The answer leaves a real limit in place. Reputational harm belongs to the same class of silent failures whose impact is difficult to quantify and predict. Teams may attempt to assign dollar values, but the speakers offer no reliable valuation method. Trust-adjusted risk can make business consequences part of the engineering discussion without making every consequence precisely measurable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
EY discusses its survey finding that 95% of leaders report AI investment and examines supporting governance and data practices.
Further reading
- Competing in the age of AIArticle
PwC discusses enterprise AI strategy and reports limited full implementation of fundamental responsible-AI capabilities.
Sahil Yadav's sponsored article covers telemetry quality, sensor drift, auditability and human oversight in network automation.
A 2025 paper by Sahil Yadav and Hariharan Ganesan discussing industrial sensors, wearable safety systems, alert fatigue and human–AI collaboration.
Full transcript
Loading transcript…