Shipping AI Into 5 Major Banks Without a 12-Month Compliance Wait — Al Chen, Cisco

Al Chen· Cisco17:41

Read the talk

Shipping AI Into 5 Major Banks Without a 12-Month Compliance Wait

Introduces the standard SaaS deployment and its potential egress risk.Open full source frame
Introduces the standard SaaS deployment and its potential egress risk.

Al Chen explains how his team turns vulnerability remediation, disaster recovery and sensitive-data handling into work that bank security teams can inspect.

From a talk by Al Chen

At a glance

Ideas worth remembering

  • CVE remediation needs an account of image versions, bank policies and upstream dependencies; a package-version change can be supplemented with an exploit test.

  • Backup and restore logs show job status; row counts and replica convergence help assess the data itself. Cross-region recovery also requires a decision about where records may reside.

  • A judge’s reasoning can expose sensitive data even when the UI is redacted. Separating scores from explanations and using a second judge on redacted input adds protection at the cost of another model call.

  • Deployment speed depends on the bank’s readiness for image approvals, GPUs, security review and access configuration as well as the vendor’s engineering.

Production requires showing the work

A bank-facing AI application can handle requests from millions of customers, making production a different proposition from a prototype or an internal productivity agent. Al Chen of Cisco describes deploying AI observability software into that environment, where a conventional SaaS connection with outbound internet access can create a route for sensitive information to leave the bank. 1:14

Chen compares compliance review to his tenth-grade math tests: the answer mattered, but most credit came from the proof. His team aims to shorten a process that can take a year to a couple of weeks by making three areas inspectable: how images reach a bank’s cluster, how data survives a regional failure, and how sensitive information stays away from unauthorized employees and outside the bank’s virtual private cloud (VPC). The timing depends on the bank’s own readiness, as the later examples make clear.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

Make CVE remediation work across banks

A single common vulnerability and exposure (CVE) can be researched in a vulnerability database. The operational problem is the multiplication: a platform has several images, each image can contain several CVEs, and banks run different image versions. Their scanners, severity policies and remediation deadlines can also differ. 2:42

Chen describes the CVE report table and its explanations for dependency-blocked fixes.Open full source frame
Chen describes the CVE report table and its explanations for dependency-blocked fixes.

Some vulnerabilities sit in transitive dependencies of open-source projects. Chen’s team follows upstream release notes and explains to bank security teams when a fix depends on a maintainer. For issues it can address, it may upgrade a package or rebuild against a base Docker image. Even then, one bank may require package version 3.8 while another prefers an older version because it sees too many CVEs in the newer one; the resulting software still has to function.

The team uses an agent skill to help research and report these cases. Its table tells engineers which CVEs to fix and gives bank security teams explanations for issues blocked by dependencies. The useful distinction is between work the team can undertake and risk it must account for while waiting on an upstream project.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:42 · section reference included

Test the exploit, not just the package version

Chen describes scanners such as Sysdig, Wiz and Prisma checking whether a package has reached a specified version. That check alone does not demonstrate whether a particular exploit still works against the application. His proposed extra evidence is a Harbor task: a harness containing agent instructions, the application’s Docker images and tests. 5:23

Chen points to the Harbor task instructions for testing the CVE.Open full source frame
Chen points to the Harbor task instructions for testing the CVE.

In the example, version 2.6.2 of a package is supposed to remediate a CVE. The agent is asked to write an exploit based on the older version, try it against the newer one, and write a test to check whether the exploit succeeds. Harbor can run trials in parallel using containers or sandboxes, giving the team repeated attempts to inspect. A failed attempt is evidence about that tested exploit, not proof that every possible exploit is absent.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:23 · section reference included

A successful restore job still needs data checks

For disaster recovery, Chen describes a ClickHouse deployment where the bank must arrange cross-region recovery itself. His pattern backs up a primary-region database to a bucket, then runs a restore job into a disaster-recovery region, which may contain several replica pods. The procedure uses clickhouse-backup, an open-source project maintained by Altinity. Chen’s assessment of managed cross-region failover is specific to the situation he describes in the talk. 7:22

Chen discusses the backup-and-restore status table and its limits.Open full source frame
Chen discusses the backup-and-restore status table and its limits.

The backup tool’s logs can show that the jobs ran, but that status does not establish what the data looks like now. A bad disk or node, a migration problem, or replication lag can leave data stale. Chen adds checks for table row counts and whether replicas within a region have converged. Those checks ask about the restored data rather than only the job’s completion.

Recovery design also makes a data-residency decision. A bucket can be placed in Oregon or Virginia, but the location of individual records can matter under jurisdictional rules. One option Chen describes is to replicate trace IDs and metadata across regions while keeping the core trace data in its permitted region. That helps preserve regional restrictions, while limiting what the remote region actually holds.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:02 · section reference included

A judge can put sensitive information back into its explanation

Redaction at an SDK, gateway or middleware can make the customer-facing UI look clean. An LLM-as-a-judge creates another path: it scores an agent’s work and produces an explanation for a human reviewer. Chen points to a seemingly innocuous instruction intended to make the explanation verifiable; if the judge sees raw traces, its reasoning may reproduce personally identifiable information (PII) or material nonpublic information (MNPI). He recommends inspecting judge prompts and traces for that leakage. 10:07

Describes suppressing sensitive reasoning and using a second judge.Open full source frame
Describes suppressing sensitive reasoning and using a second judge.

The proposed flow separates the score from the reasoning. The score persists for evaluation. Application code checks whether a record contains sensitive information: reasoning can persist when the check finds none, and is suppressed when it finds sensitive information. The diagram shows those branches and the alternative for cases where a bank still wants an explanation.

That alternative runs a second judge on input with sensitive information redacted and displays its explanation. It costs another LLM call on the same request. Chen presents redacted input as a reason for confidence in the second output; the protection depends on the redaction actually covering the sensitive information.

Separate the score from explanations

Judge on raw traceraw — Produces a score and reasoning.
Score persistsraw — Available in the UI for evaluation.
Sensitivity checkraw — Application code checks the record for sensitive information.
Show reasoningraw — Persist reasoning when the check finds no sensitive information.
Suppress reasoningraw — Do not show the raw-data judge’s reasoning when the check finds sensitive information.
Redacted inputalternative — Sensitive information is removed before the second judge sees it.
Second judgealternative — Generates an explanation for display using the redacted input.
The raw-data judge supplies a score; its reasoning follows a sensitivity check. A second judge can generate an explanation from redacted input.
Read the diagram as text
  • Judge on raw trace. raw — Produces a score and reasoning.
  • Score persists. raw — Available in the UI for evaluation.
  • Sensitivity check. raw — Application code checks the record for sensitive information.
  • Show reasoning. raw — Persist reasoning when the check finds no sensitive information.
  • Suppress reasoning. raw — Do not show the raw-data judge’s reasoning when the check finds sensitive information.
  • Redacted input. alternative — Sensitive information is removed before the second judge sees it.
  • Second judge. alternative — Generates an explanation for display using the redacted input.
  • Judge on raw trace → Score persists: Score.
  • Judge on raw trace → Sensitivity check: Reasoning.
  • Sensitivity check → Show reasoning: No sensitive information found.
  • Sensitivity check → Suppress reasoning: Sensitive information found.
  • Redacted input → Second judge: Input.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:07 · section reference included

The deployment clock depends on the bank

A vendor’s promise of deployment in a few weeks means little until it is checked against the bank’s infrastructure, security posture and cluster. Chen looks for practical signals: is there already a process for pulling images from a registry, and are GPUs sourced and ready for inference? 13:06

The timeline can extend to months when formal security reviews are needed or the bank is building a custom LLM gateway or role-based access control configuration. It can stretch further if image-pull approval and GPU procurement have no established process. Chen finds that many of these delays come from organizational alignment as much as technology.

His team treats a customer’s problem as its own, dividing the work among engineering, workflow, and human and organizational streams, then automating where it can. The three pillars in the talk sit within a larger production checklist. The stakes explain the methodical pace: checking accounts, mortgages and credit cards leave little room for an answer that merely looks plausible.

Chen closes with the other half of his school-test analogy. After a test, he and his friends reconstructed problems at the lunch table and compared how they solved them. He offers the conference the same role: a place to exchange approaches to difficult customer problems, including the slow, messy organizational work that makes production possible.

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:06 · section reference included

Resources

From the talk

  • A way to follow up on Chen’s invitation to discuss deployment pillars and his team’s production checklist.

Transcript download
Full transcript

Loading transcript…