AI Engineer NYC 2026 Online Talks
Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium
Slide chapters
23 slide and demo groups · Timestamps are approximate. 2 additional groups were omitted from this pass.
Read the talk
Your GPU Neighbor Can Hear Your Model
Branislav Brzak traces an H100 timing side channel from reads of an attacker’s own memory to model and workload fingerprints, then shows what changes across NVLink and what operators can observe.
From a talk by Branislav Brzak
At a glance
Ideas worth remembering
MIG partitions compute and memory, but GPU-wide memory barriers can still change the latency of another tenant’s reads of its own memory.
Token tempo reveals decoding speed; within-token timing structure adds architectural information when similar speeds make models hard to distinguish.
The demonstrated workload inference depends on speculative decoding: differing draft acceptance changes step timing without exposing the actual generated text.
The separate-GPU NVLink experiment preserves enough tempo information for a useful model fingerprint, although layer detail falls below the noise.
Tenant placement, inter-GPU communication requirements and monitoring of repeated GPU probes are separate operator decisions.
A GPU-wide barrier slows reads in another slice
MIG isolates compute resources, memory capacity, bandwidth quotas and faults. It also partitions the L2 cache: one instance cannot evict another instance’s cache entries. The remaining connection is a GPU-wide memory barrier, or membar. A barrier issued by one instance affects every L2 partition on the card. Consequently, a tenant can experience slightly slower reads of its own memory while another tenant is busy.
The probe turns that slowdown into a repeated measurement: load from a small buffer owned by the attacker, time the load, and repeat every five microseconds. No step requires a read from the victim’s memory. Brzak credits the researchers behind Behind Bars at USENIX Security for both this probing primitive and the underlying MIG memory-barrier leak.
How much does the neighboring workload change the measurement? The slide shows medians of 715.9 ns idle and 719.1 ns while serving Qwen3-8B, labeling the gap about 3 ns; the spoken account says microseconds. That small shift is useful because the attacker records a time series, preserving when slowdowns occur and how they repeat. The signal comes from the structure of those measurements, beyond the difference between two medians.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Token rhythm gives speed; peaks within tokens give structure
With an eight-billion-parameter model running on the victim slice, the trace shows one burst of GPU activity per generated token. Frequency analysis of the attacker’s latency measurements finds a fundamental at 81 Hz: the recurring token-generation cycle corresponds to 81 tokens per second. Harmonics accompany it because the work inside each token has its own structure, rather than forming one smooth pulse.
What does that structure look like inside a token? The demonstrated time-domain trace contains 39 bumps. Green peaks mark layer invocations; orange peaks mark additional processing between layers. The distinction matters: counting every visible bump as a model layer would overestimate depth. The trace connects two scales of observation—the spacing between tokens and the repeated work within each token.
Token rate is useful but ambiguous. Two eight-billion-parameter models in the comparison decode only 1.6% apart in speed. A tempo measurement can narrow the possibilities while leaving those models nearly indistinguishable. The next step therefore looks for the layer-related structure within the timing signal.
A fold-and-average procedure brings out recurring peaks. Across 22 traces, the resulting count lies a mean distance of only 0.028 from an integer. Brzak uses this near-integer result as evidence that the pattern follows discrete model operations: a fixed hardware frequency divided by varying token rates would not generally land so close to whole numbers. The observed counts correspond to model depth plus one to three extra peaks, retaining the distinction between layers and other processing.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Fit candidate depths without a labeled training set
The depth estimator is a small best-fit procedure, described as fewer than 30 lines of code. It checks candidate depths from 16 through 48, measures how strongly each candidate’s expected signature appears in the recorded trace, and selects the strongest fit. It needs no labeled training set. Tempo and harmonic structure supply the evidence used to compare candidates.
How decisive is that comparison? In the presented eight-billion-parameter example, the 36-layer candidate wins clearly over competing depths. Because the procedure fits a structural signature rather than looking up a previously labeled trace, Brzak proposes that it can estimate depth for unseen models, including proprietary or unreleased ones. Recovering depth alone does not supply the name of an unknown model; it reveals an architectural property that can help fingerprint it.
The estimator also produces a confidence value. In the reported testing, the only two incorrect answers were the two least confident ones. Confidence therefore helped identify the uncertain fits in that test, although the recording does not establish a confidence threshold that operators could apply universally.
Related models make the task harder. The comparison includes Llama 3 8B, Llama 3.1 8B and DeepSeek’s R1-Distill Llama 8B, all sharing a 32-layer backbone. Their common architecture preserves much of the timing signature, so depth alone cannot separate them. Brzak reports a median per-layer latency shift of only 1.6 nanoseconds and says the classifier nevertheless distinguishes the models. This is the demanding case: the broad structural fingerprint stays similar while the distinguishing timing differences become very small.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Speculative decoding makes text categories affect timing
The next experiment keeps the timing channel and changes the question: is the model producing prose, code, JSON or a retrieval response? It issues a verdict every half second using the preceding five seconds of measurements. Of 40 verdicts, 31 are correct; majority voting correctly identifies all four complete responses in the demonstration. Comparing the predictions with the actual workload would show both the short-window mistakes and why a response-level vote can still succeed.
This result depends on speculative decoding, enabled for the experiment and described here as an opt-in feature in vLLM. Speculative decoding drafts tokens and accepts some number of them at each step. Text predictability changes how many drafts are accepted; that changes step timing. The probe observes this timing consequence and uses it to infer a workload category. It still does not read the generated text.
The tests span 11 configurations, including dense and mixture-of-experts models, with BF16 and FP8 precision. The workload-dependent signature appears throughout, but its strength varies sharply: the reported spread ranges from 70.2% in one configuration to 0.9% in another. Prose runs fastest in 10 of the 11 configurations. That repeated ordering helps explain why category inference can work, while the wide variation cautions against expecting the same separation everywhere.
The inference thus has two useful conditions: an attacker already has information about the model, and speculative decoding is active. Under those conditions, acceptance behavior adds another source of timing variation. The demonstrated workload classification should not be extended to ordinary decoding configurations without that mechanism.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move to another GPU and the token rhythm survives
The second setup removes the shared physical die. An eight-H100 server connects its GPUs through an NVSwitch fabric. The victim runs tensor parallel inference across GPUs 0 and 1; the attacker runs on GPU 2. Peer access is an intended feature used by multi-GPU jobs. This tests a different deployment arrangement from MIG: in the setup described, MIG’s isolation rules prevent the peer communication needed for tensor parallelism.
The attacker now pointer-chases memory located on GPU 0 over NVLink, timing each hop from GPU 2. An idle hop takes about 890 nanoseconds; during victim activity, measured latency climbs to as much as 300 microseconds. What survives the extra distance? In the three half-second captures described, token bursts remain visible enough to count by eye, while the layer-related detail falls below the noise.
The topology below answers why a separate GPU is still close enough to observe the workload: the probe reaches memory on a GPU participating in the victim’s job. The victim’s two-GPU placement and the attacker’s remote access are separate relationships; the diagram does not imply that the attacker reads the victim’s allocated data.
With five models served under otherwise identical conditions, the remote method achieves 79.2% classification accuracy against a 20% chance baseline. Its failures follow the missing information: two eight-billion-parameter models decoding only 3 Hz apart blur together when tempo is the principal remaining fingerprint. The stronger MIG trace had additional within-token structure to help separate such cases. Losing detail weakens the attack without removing its useful signal.
Remote probing on the same NVSwitch-connected server
Read the diagram as text
- Victim inference job. victim — Tensor parallel execution across two GPUs.
- GPU 0. server — Participates in inference and hosts the memory reached by the remote probe.
- GPU 1. server — Also participates in the victim’s tensor parallel job.
- GPU 2: attacker. server — Runs the pointer chase and measures remote-access latency.
- Victim inference job → GPU 0: Executes part of the tensor parallel job.
- Victim inference job → GPU 1: Executes part of the tensor parallel job.
- GPU 2: attacker → GPU 0: Times pointer-chase accesses over NVLink / NVSwitch.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The probing workload has a GPU-visible signature
The attack is quiet in transferred data but conspicuous in its execution pattern. In Brzak’s demonstration, Stealthium’s kernel monitoring catches a single kernel relaunched about 3,000 times, repeatedly accessing the same 40 KB of memory. Only 1 KB returns to the CPU. Those observations fit the probe’s purpose: repeated access creates timing measurements, so a large conventional output is unnecessary.
Brzak reports detection in under 600 milliseconds, with fewer than 50 tokens of the response produced. That is a result for Stealthium’s demonstrated monitoring and alert pipeline, not an established detection guarantee for every variant. The useful operational detail is the combination of repeated kernel launches, a fixed small memory region and little CPU-bound output: GPU execution behavior exposes activity that output-volume monitoring alone could miss.
The closing recommendations address three distinct decisions:
- Tenant placement. Avoid placing an untrusted tenant’s MIG slice beside a sensitive model. Placement controls whether the attacker gains the proximity used by the first experiment.
- Communication needs. MIG rules out peer-to-peer communication in the arrangement discussed, while tensor parallel inference uses inter-GPU communication. The tested NVLink arrangement retains a timing fingerprint but loses usable layer detail; that tradeoff does not make it free of leakage.
- GPU observation. Monitor workloads where the probe runs. Kernel launches and memory-access behavior provide evidence of repeated timing probes even when very little data returns to the CPU.
The recording ends by returning credit to Behind Bars. Its memory-barrier primitive supplies the starting point; these experiments develop what model-serving activity can reveal through it and through a separate NVLink probe. Physical memory isolation remains useful, but the timing of shared hardware operations needs its own examination.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
21 slide groups · 2 demo milestones. Extracted from the recording; expand builds to follow changes. Timestamps are approximate. 2 additional groups were omitted from this pass.
Your GPU neighbor can hear your model
A co-tenant sharing a cloud GPU can watch timing to learn which model is running, how fast it responds, and what kind of work it is doing, without touching its data.
Text in image
Your GPU neighbor can hear your model What a co-tenant learns about your LLM from timing alone STEALTHIUM AI ENGINEER NYC • OCTOBER 2026 Branislav Brzak Security Researcher
- 0:01
Hello, everyone. This talk is about the danger of your LLMs information leaking through shared GPU hardware. If you're running a model in the cloud on a GPU you share with other tenants, someone next to you can learn things about it. They can tell what model you're running, how fast it's responding, and even what kind of work it's doing right now, all without touching your data, just by watching timing. That's what this talk covers, what a co-tenant on the same hardware can learn about your LLM from timing alone, and what you can do about it. To start, I'm
Your GPU neighbor can hear your model
A co-tenant sharing a cloud GPU can watch timing to learn which model is running, how fast it responds, and what kind of work it is doing, without touching its data.
Text in image
Your GPU neighbor can hear your model What a co-tenant learns about your LLM from timing alone STEALTHIUM AI ENGINEER NYC • OCTOBER 2026 Branislav Brzak Security Researcher
We watch what runs on your GPUs
Stealthium turns CUDA kernel launches into telemetry, flags workloads that behave like attacks, and offers observability from individual nodes to multi-cluster fleets. Brzak focuses on low-level GPU security research.
Text in image
WHO WE ARE We watch what runs on your GPUs See inside every GPU, every AI workload. At runtime, anywhere. 01 Runtime telemetry Every CUDA kernel launch on every GPU, turned into data points 02 Security Flags GPU workloads that behave like attacks, while they run 03 Observability From a single node to multi-cluster fleets Branislav Brzak Security Researcher Cybersecurity researcher at Stealthium, working down at the level of GPU internals. 2
- 0:31
Branislav Brzak, a cybersecurity researcher employed at Stealthium. At Stealthium, we specialize in securing and providing observability into the environments AI systems run on. My area of focus is low-level GPU-related security research. That's something this talk covers. When you share a GPU between tenants, you would expect full isolation, but the isolation provided may not prove to be sufficient. In the case of this attack, the attacker doesn't even need any special permissions like root access. It doesn't
Share a GPU, and you are not properly isolated
GPU slicing isolates space, but shared hardware still leaks timing: a neighboring tenant can time its own reads to learn about a model without root access or a traditional exploit.
Text in image
THE CLAIM Share a GPU, and you are not properly isolated Cloud GPUs get sliced between tenants. The slicing isolates space. It does not isolate time. The attacker needs no permissions. No root. No driver bug. No byte of your memory. An ordinary tenant, running an ordinary CUDA kernel, timing how long its own reads take. 3
- 1:01
need any traditional exploit either. The only thing it does is time the parts of the hardware that are still shared among tenants. This is a demonstration video of what the victim's and the attacker's dashboards look like. On the left, we have Q* model doing inference. On the right, the attacker uses a side channel that Q* leaks into as it's running. From that, it determines exactly what model it is and what kind of text it's producing right now.
Timing demo: candidate models
As Tenant A begins a cache explanation, Tenant B measures its own read latency and compares candidate models. Step time alone makes two look alike; the work classifier is still reading the signal.
Text in image
TENANT A · WHAT ITS MODEL IS WRITING the neighbor across the slice never sees this Caching trades memory for speed. An LRU cache keeps the entries you touched most recently and throws out the one you h TENANT B · WHAT IT MEASURES its own memory read latency, nothing else WHICH MODEL STEP TIME ALONE: TWO LOOK ALIKE Qwen3-8B Llama-3-8B GLM-4-9B Qwen2-7B WHAT KIND OF WORK reading the signal...
1:11Demo milestone
GPU side-channel demonstration
As tenant A writes Python cache code, tenant B measures its own memory read latency. The demo identifies the model as Qwen3-8B and the work as code.
Text in image
TENANT A · WHAT ITS MODEL IS WRITING the neighbor across the slice never sees this Caching trades memory for speed. An LRU cache keeps the entries you touched most recently and throws out the one you have not used for the longest time. Here is a small version in Python: from collections import OrderedDict class LRUCache: def __init__(self, capacity): self.capacity = capacity self.data = OrderedDict() def get(self, key): if key not in self.data: return None TENANT B · WHAT IT MEASURES its own memory read latency, nothing else WHICH MODEL IDENTIFIED FROM TIMING ALONE Qwen3-8B step rate, then layer count WHAT KIND OF WORK A VERDICT EVERY 0.5 s CODE matches what tenant A is doing1:18Demo milestone
Show 2 builds
- 1:18
A cache code example appears beside a timing trace, a Qwen3-8B identification, and a code-work verdict.
Text in image
TENANT A · WHAT ITS MODEL IS WRITING the neighbor across the slice never sees this Caching trades memory for speed. An LRU cache keeps the entries you touched most recently and throws out the one you have not used for the longest time. Here is a small version in Python: from collections import OrderedDict class LRUCache: def __init__(self, capacity): self.capacity = capacity self.data = OrderedDict() def get(self, key): if key not in self.data: return None TENANT B · WHAT IT MEASURES its own memory read latency, nothing else WHICH MODEL IDENTIFIED FROM TIMING ALONE Qwen3-8B step rate, then layer count WHAT KIND OF WORK A VERDICT EVERY 0.5 s CODE matches what tenant A is doing - 1:23
A model writes Python cache code while a neighboring tenant measures latency and identifies Qwen3-8B and code.
Text in image
TENANT A · WHAT ITS MODEL IS WRITING the neighbor across the slice never sees this Caching trades memory for speed. An LRU cache keeps the entries you touched most recently and throws out the one you have not used for the longest time. Here is a small version in Python: from collections import OrderedDict class LRUCache: def __init__(self, capacity): self.capacity = capacity self.data = OrderedDict() def get(self, key): if key not in self.data: return None self.data.move_to_end(key) return self TENANT B · WHAT IT MEASURES its own memory read latency, nothing else WHICH MODEL IDENTIFIED FROM TIMING ALONE Qwen3-8B step rate, then layer count WHAT KIND OF WORK A VERDICT EVERY 0.5 s CODE matches what tenant A is doing
- 1:18
A stranger on your GPU can tell
GPU timing reveals model architecture and depth (14 of 16 models exact), decode speed, activity and work type. The H100 measurements do not reveal prompts or answers.
Text in image
THE SHORT VERSION A stranger on your GPU can tell 01 Which model Architecture and depth, 14 of 16 models exact 02 How fast Decode speed, within 99.3% of the server’s own figure 03 When, and how much Busy periods, tokens and answer length 04 What kind of work Code, JSON, prose or retrieval It does not learn your prompts or your answers. Measured on NVIDIA H100: MIG slices of one GPU, and separate GPUs over NVLink 5
- 1:30
These are just the headline claims, but not the whole story. We can tell which model it is, architecture, and depth within ninety-seven percent accuracy. We can tell how fast is the coding within ninety-nine point three percent accuracy. We can tell when and how much, the busy periods, token counts, and the answer length. What we can tell what kind of work it's doing, code, joy, prose, JSON, or retrieval. What we cannot tell, though, are the actual prompts, answers, or contents, just timing,
- 2:00
and all of these on real H100s serving real LLMs. If you're using NVIDIA hardware, GPU isolation is achieved using Multi-Instance GPU, or MIG for short. Each isolated MIG slice has its own physical compute units and dedicated VRAM. That provides hardware isolation on those parts. In our example, the LLM we are serving runs on a slice with three compute units and forty gigabytes of VRAM. The attacker runs on just one compute units and ten gigabytes
One GPU, sliced into tenants
An H100 places the LLM and an attacker in separate MIG slices, but the shared memory fabric still leaks timing.
Text in image
PART 1 • HARDWARE ISOLATION One GPU, sliced into tenants ONE H100 GPU Tenant A Serving an LLM 3g.40gb Tenant B Anything 1g.10gb C 1g D 1g Free 1g Memory fabric is still shared The hardware leaks the timing NVIDIA Multi-Instance GPU (MIG) on H100 80GB: up to seven isolated instances 6
- 2:30
of VRAM.
MIG isolates space, not time
MIG partitions compute and memory resources, but a GPU-wide memory barrier still reaches every L2 partition. When one tenant is busy, another tenant’s own reads can slow down.
Text in image
PART 1 • HARDWARE ISOLATION MIG isolates space, not time Isolated • Compute units • Memory capacity • Bandwidth quotas • Faults and errors Still shared • Memory barriers • Timing A GPU-wide memory barrier takes effect in every L2 partition on the card 7
- 2:33
What's isolated are compute units, memory capacity, bandwidth quotas, and faults or errors. The L2 cache is partitioned per instance too. There is no cross-instance eviction, so it's not shared in that sense. The actual leaks are memory barriers or membars, which are GPU wide. One instance's barrier still touches every L2 partition on the card. Timing is the one thing that isn't isolated. So when one tenant is busy, the other tenants' own reads gets slightly slower.
- 3:03
The key point is the, a GPU-wide memory barrier takes effect in every L2 partition on the card. The next slide builds the whole attack on that one fact.
It reads the clock
An attacker uses 12,288 threads to time loads from its own 1.5 MB buffer every 5 µs for seconds. The buffer stays in L2; the attacker never reads the victim’s memory.
Text in image
PART 2 • THE SIDE CHANNEL It never reads your memory. It reads the clock. Load from its own small buffer Time how long that load took Repeat every 5 µs, for seconds 12,288 threads 1.5 MB buffer, stays in L2 5 µs per sample Primitive from Behind Bars Gu, Levine, Zhang, Sorensen, Guo • USENIX Security 8
- 3:16
The vector is that the attacker only ever reads its own memory. It only times its own reads. It never touches the victim's memory. There are three steps: load from its own small buffer, time the load, and repeat every five microseconds. This primitive and the MIG membar leak itself isn't our discovery. Credit goes to the re- researchers behind Behind Bars at USENIX Security. A big thanks for their work.
- 3:44
Now try looking at the timing differences. With the naked eye comparing your tenant idle versus serving an LLM, you wouldn't even be able to discern them. The median latency is only three microseconds higher. To the attacker, though, those three microseconds are all it needs. That's enough to gather all the information we already talked about.
Look at it as a frequency
GPU activity pulses once per token while Qwen3-8B runs on the victim slice. The attacker’s latency spectrum reveals an 81 Hz token rate and higher harmonics.
Text in image
PART 2 • THE SIDE CHANNEL Look at it as a frequency GPU ACTIVITY 81 Hz • one pulse per token 81 Hz = tokens per second Each token is comprised of distinct frequencies Frequency ticks: 100 Hz, 200 Hz, 300 Hz, 400 Hz, 500 Hz Attacker’s latency spectrum, whitened • Qwen3-8B on the victim slice 10
- 4:06
Q* with eight billion parameters is running on the victim slice throughout this one. All captured data was taken from a live model. At the top is GPU activity over time, one burst per token. Below is the frequency spectrum of the attacker's own latency trace. There is a fundamental at eighty-one hertz, which is just tokens per second plus harmonics since each token is a mix of distinct frequencies, not one clean one.
Zoom into one of those pulses
A 12.4 ms token trace, averaged across 807 Qwen3-8B tokens, shows 36 layer peaks and three weaker peaks associated with overhead, embedding, norm and the output head.
Text in image
PART 2 • THE SIDE CHANNEL Zoom into one of those pulses Every token is one pulse, 12.4 ms wide. Inside it, the layers beat too. 39 beats per token, the real trace, in time QWEN3-8B • 807 REAL TOKENS AVERAGED Chart labels: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 +3 28 29 30 31 32 33 34 35 36 36 mint dots, one bump per transformer layer, each a real local peak of the trace above. 3 amber dots, weaker but still real: fixed overhead, embedding, norm and the output head. 11
- 4:33
This is a real trace, one token in the time domain. The token shows a total of thirty-nine bumps. The bumps with green peaks are the layer invocations, while orange ones are additional between layer processing. In the later slides, we will demonstrate how we determine the actual layer counts, even when those extra bumps are present. Here we look at the decode speed itself. Just by timing, we found the token rate completely blind. So speed alone is something an
Every model keeps its own tempo
Timing recovered token rates within 0.7% of the server’s figure on 19 of 22 traces. Similar speeds can belong to different models, so speed alone is not a clean fingerprint.
Text in image
PART 2 • THE SIDE CHANNEL Every model keeps its own tempo Qwen2-7B 90.9 tokens/s Llama-3-8B 82.2 tokens/s GLM-4-9B 67.4 tokens/s gpt-oss-20B 32.8 tokens/s Token rate found blind, within 0.7% of the server’s own figure on 19 of 22 traces 12
- 5:03
attacker can recover accurately. But speed by itself isn't a clean fingerprint. Q* eight billion and Llama 3 eight billion decode only one point six percent apart in their token rate. So from speed alone, you'd think they're the same model. It narrows things down, but it doesn't nail it. We need something sharper, layer counts. That's what the next few slides get into. We use the same fold and average trick from, from before. This time, we run it once per model instead of once per
The peak tells most of the story
The traces show beat counts close to each model’s layer count plus two or three peaks. Across 22 traces, the mean distance from a whole number is 0.028.
Text in image
PART 4 • LAYER EXTRACTION The peak tells most of the story The main thing the attacker uses is tiny spikes that allow it to infer a lot of facts Qwen2-7B 28 layers 30 beats = 28+2 Llama-3-8B 32 layers 34 beats = 32+2 Qwen3-8B 36 layers 39 beats = 36+3 GLM-4-9B 40 layers 43 beats = 40+3 Same real technique as the Qwen3-8B close-up: one token, folded and averaged. Mean distance of the beat count from a whole [illegible] across 22 traces: 0.028. 15
- 5:33
token. What comes out is a bit count that lands almost exactly on a whole number. The mean distance from an integer across twenty-two traces is just zero point zero two eight. That's the tell. If these were just a fixed hardware artifact divided by different token rates, we'd expect fractions, not whole numbers. Each count works out to the model's layer count plus one to two-- one to three additional peaks.
Test the usual depths, keep the fit
The method checks how strongly each candidate depth’s signature appears in the Qwen3-8B trace. The 36-layer candidate fits far better than the others, without training, labels, or configuration.
Text in image
PART 4 • LAYER EXTRACTION Test the usual depths, keep the fit Real models come in a handful of common depths. Try each one against the real signal and see which one fits. 16 layers 20 layers 24 layers 28 layers 32 layers 36 layers 40 layers 48 layers 36 layers fits far better than any other guess. That is the depth. How strongly each guess’s signature actually shows up in the real Qwen3-8B trace. No training, no labels, no config. 16
- 6:03
Every model keeps its own signal fingerprints, a mix of tempo and harmonic structure. To be clear, this is not a trained machine learning classifier. There's no training set, no labels, no configuration. It's under thirty lines of code. The method checks each candidate's depth from sixteen to forty-eight. It measures how strongly that depth's expected signature shows up in the real trace. Then it picks whichever fits best. For Qwen three eight billion, thirty-six layers wins clearly by four point times
- 6:32
over the next candidate. Because this is a best-fit test, not a lookup table, it also works on models it never seen before. That includes proprietary on unreleased ones, since there's nothing it needs to have seen in the first place.
It counts your layers
The layer-fit classifier identifies the exact depth of 14 of 16 models blind. Its two misses are its least confident calls, and both models have fewer than 2B parameters.
Text in image
PART 4 • LAYER EXTRACTION It counts your layers Qwen3-8B — 36 layers GLM-4-9B — 40 layers Qwen2-7B — 28 layers Llama-3-8B — 32 layers Qwen2.5-7B — 28 layers Llama-3.1-8B — 32 layers R1-Distill-8B — 32 layers gpt-oss-20B — 24 layers Qwen1.5-MoE — 24 layers OLMoE-1B-7B — 16 layers Granite-3.0-3B — 32 layers Granite-3.0-1B — 24 layers Llama-2-7B — 32 layers Granite-3.1-1B — 24 layers Qwen2.5-1.5B — 16 layers Qwen2.5-0.5B — 48 layers CONFIDENCE 14/16 models to the exact layer, blind The two misses are its two least confident calls. Both are under 2B parameters. 17
- 6:52
And this works surprisingly well. The layer fit classifier also outputs the confidence level. Across our testing, only two least confident answers ended up being wrong ones.
A fine-tune doesn’t hide its base
Three related models have average per-layer timings within 1.6 ns. Their signals look alike, yet the classifier can still distinguish them.
Text in image
PART 4 • LAYER EXTRACTION A fine-tune doesn’t hide its base Three related models look different on paper. On the signal, only slightly different, but not really. Legend: Llama-3-8B 686.76 ns Llama-3.1-8B 686.38 ns R1-Distill-Llama-8B 685.20 ns Y-axis ticks: 690, 688, 686, 684 Average lines: 686.76 ns, 686.38 ns, 685.20 ns Solid lines: each model’s own average. Within 1.6 ns of each other. STE[illegible] 18
- 7:05
This same method also picks up fine-tunes and distillations that, uh, that share the same base model. That's because the architecture and layer count carry over unchanged. Take three related models as an example. Llama three eight billion from Meta has thirty-two layers. Llama three point one eight billion, also from Meta, uses that same thirty-two-layer backbone with minor upgrades. And R1-Distill Llama eight billion from DeepSeek is an improvement on top of that. The
- 7:35
per-layer timing comes out to only one point six nanoseconds median latency shift of each other. The rest of the signal looks the same. Regardless of this, our classifier is still able to discern between them. Now we ask a different question on the same channel, not which model, but what it's doing right now. We look at four workload types, prose, code, JSON, and retrieval. We give a verdict on every half second based on the last five seconds of timing. Out
It reads the work, not the words
A timing classifier labels four workload types every half-second using the previous five seconds. It got 31 of 40 verdicts right and identified all 4 responses by majority vote, with speculative decoding on.
Text in image
PART 5 • WORKLOAD DETECTION It reads the work, not the words RESPONSE SHOWN FOR LEGIBILITY, VERDICTS ARE MEASURED An LRU cache keeps recent entries and evicts the oldest. Prose ✓ def get(self, key): Code ✓ if key not in self.cache: JSON ✗ return None Code ✓ return self.cache[key] Retrieval ✗ def put(self, key, value): Code ✓ {"id": 14, "sku": "A123", "price": 15.99} JSON ✓ 31 of 40 half-second verdicts right, chance is 10 4 of 4 whole responses named correctly 19- 8:05
of forty half-second verdicts, thirty-one were right, and four full responses were collect, correctly identified by mar-majority votes. This is the same classifier from the video earlier, misses included. One condition is worth stating. This was measured with speculative decoding turned on. That's an opt-in in vLLM, not the default. Here's the mechanism. How predictable the text is changes how many drafted tokens get accepted at each step. That
Not one model: eleven setups
With speculative decoding on, workload step-time spread ranges from 17.2% to 0.9% across 11 setups; 10 of 11 run prose fastest. Text predictability changes how many drafted tokens are accepted per step.
Text in image
PART 5 · WORKLOAD DETECTION Not one model: eleven setups gpt-oss-20B 17.2% Granite-3.0-1B 6.2% Qwen1.5-MoE 6.0% OLMoE-7B 3.9% Qwen2.5-7B 3.5% Qwen3-8B 3.2% Granite-3.1-1B 2.1% Qwen3-32B fp8 1.1% Llama-3.1-8B 1.1% Qwen3-8B fp8 1.0% GLM-4-9B 0.9% SPREAD IN STEP TIME BETWEEN WORKLOAD TYPES 10 of 11 ran prose fastest. Chance is 1 in 4. Measured with speculative decoding on. vLLM ships with it off. [illegible] rows: fp8 weights 20
- 8:35
changes step time, and that's the signal we're picking up. We tested eleven configurations, dense and mixture of experts, BF16 and FP8. The same fingerprint showed up every time, just at very different strengths. Seventy point two percent spread on GPT-O6 twenty billion versus zero point nine percent on a GLM four nine billion. Ten out of eleven run prose fastest when the chance alone would only give you one in four. So say the attacker already knows the model from the earlier
- 9:05
slides, and speculative decoding happens to be on. It can also tell what kind of text is coming out. Now we take the shared chip out of the picture entirely. MIG needs the attacker on the same physical die. This attack doesn't. The setup here is an eight times H100 server connected over an NVSwitch fabric. The victim runs tensor parallel across GPU zero and one. The attacker just needs any other GPU in that same server,
Now take the shared GPU away
Tenant A splits its model across GPUs 0 and 1. An attacker on GPU 2 can use the same NVSwitch fabric from another GPU in the server, without sharing a GPU or using MIG.
Text in image
PART 6 • ACROSS NVLINK Now take the shared GPU away ONE SERVER, 8 × H100 GPU 0 Tenant A GPU 1 Tenant A GPU 2 Tenant B GPU 3 others GPU 4 others GPU 5 others GPU 6 others GPU 7 others NVSwitch fabric Tenant A’s model is split over GPU 0 and 1 No MIG. The attacker only needs another GPU in the same server. 22
Now take the shared GPU away
Tenant A splits its model across GPUs 0 and 1. An attacker on GPU 2 can use the same NVSwitch fabric from another GPU in the server, without sharing a GPU or using MIG.
Text in image
PART 6 • ACROSS NVLINK Now take the shared GPU away ONE SERVER, 8 × H100 GPU 0 Tenant A GPU 1 Tenant A GPU 2 Tenant B GPU 3 others GPU 4 others GPU 5 others GPU 6 others GPU 7 others NVSwitch fabric Tenant A’s model is split over GPU 0 and 1 No MIG. The attacker only needs another GPU in the same server. 22
- 9:34
GPU two in this case. Peer access over NVSwitch is a real intended feature. It's how multi-GPU jobs normally work, not a bug being abused. MIG tensor parallelism also can't coexist on the same device, so this is really a different population of deployments, not just the same attack repeated.
- 9:58
The probe runs on GPU two. It pointer chases memory that actually is on GPU zero over NVLink, timing every hop along the way. An idle hop takes about eight hundred and ninety nanoseconds. Once the victim starts working, that climbs up to three hundred microseconds. These are three half-second me-captures, and you can actually count the tokens by eye. What we lose compared to MIG is the layer
- 10:27
bit is below the noise at this distance. So over NVLink, the fingerprints we-we're left with is mostly tempo, not layer counts. Here, we test the five models on identical serving setup. Only the model itself gives away which one it is. This method gives seventy-nine point two percent accuracy. Now, it fails exactly where you'd expect the tempo to fail. Llama three point five eight billion and R1-Distill are both eight billion
It names the model, one GPU over
Across NVLink, model timing identifies five models with 79% accuracy against 20% chance. Llama and DeepSeek often blur together because their decode speeds are similar.
Text in image
PART 6 • ACROSS NVLINK It names the model, one GPU over Columns: Llama | DeepSeek | GLM-4 | gpt-oss | Qwen3 Llama-3.1-8B: 64% | 36% | · | · | · DeepSeek-R1-8B: 54% | 11% | 32% | 4% | · GLM-4-9B: 5% | 5% | 79% | 10% | · gpt-oss-20B: 1% | 1% | 4% | 93% | · Qwen3-32B: · | · | · | · | 100% 79% right, against 20% chance Llama and DeepSeek decode at the same speed, so they blur together. 5 models, identical serving setup, 0.558 s per reading, drift-controlled 24
- 10:57
models that decode three hertz apart. So they blur together, where MIG's layer counts would have separated them easily. The takeaway is that the NVLink's signal-to-noise ratio is wor-worse than MIG's, since it loses the layer bit. But worse signal-to-noise ratio isn't the same as safety. Seventy-nine percent against a twenty percent baseline is still a real fingerprint, not noise. First, some good news. Our own product at work caught the attack
It never looked like real work
Stealthium’s alert flagged a probe that relaunched one kernel 3,173 times and sent 1,172 B back in 590 ms. The speaker says the timing itself carried the payload.
Text in image
It never looked like real work PART 7 • DEFENSE A real workload computes and moves results. A probe only measures. 3,173 — kernel relaunches 1,172 B — sent back, in total 590 ms — start to finish < 48 tokens — worth of reply time Interface text: High ALERT-1770 GPU-based covert channel / C2 Take Action Export Alert summary Resource details GPU-based covert channel / C2 Process PID=100216 (primitive1_ptx) relaunched a single kernel [illegible] 3173 times while holding a negligible device working set (14344 bytes) and moving only 1172 bytes to/from the host. Carrying data by modulating GPU contention this way is a covert channel: it leaves over GPU timing, not the network, so a packet monitor sees nothing. The process is not on the approved GPU workload list. Process name: primitive1_ptx Thu, Aug 06, 13:13:41 Gpu Covert Channel Primitive1_ptx Relaunched [illegible] 3173 Times, 14344-Byte Working Set, 1172 Bytes Host I/O Event Type: gpu covert channel Command line: [illegible] Args: [illegible] Working dir: [illegible] 25
- 11:27
straight out of the box. Stealthium monitors every single kernel launch across every GPU, analyzes suspicious workloads, and runs them through a full alert pipeline, which is something not every vendor can do. Here is what's made this one stand out. A single kernel relaunched a-about three thousand times, hitting the exact same forty ki-kilobytes of memory over and over instead of moving to new data. Only one kilobyte of data was sent back to the CPU the entire time, because the
- 11:57
timing itself was the payload, not the output. In real terms, that is under six hundred milliseconds start to finish with fewer than fifty tokens of a reply. We caught it before the response was ever half typed.
What to do about it
Keep sensitive models away from untrusted MIG tenants. MIG rules out tensor parallelism; sharding over NVLink limits leakage to timing. Watch for probes that relaunch while holding and sending back little data.
Text in image
PART 7 • DEFENSE What to do about it 01 Place with care Don't put sensitive models on a MIG slice next to tenants you don't trust. 02 Know the tradeoff MIG and tensor parallelism can't share a device. TP over NVLink leaks less, tempo only. 03 Watch the GPU itself A probe looks nothing like a workload: it relaunches, holds nothing and sends nothing back. 27
- 12:14
So three takeaways. First, placement is something operators actually control. So do not put an untrusted tenant's MIG slice right next to sensitive model. Second, know the trade-off. MIG rules out peer-to-peer communication, meaning there is no tensor parallelism on MIG. Sharding over NVLink instead limits what leaks to timing alone rather than layer counts. Third, watch the GPU itself. Make sure the host
- 12:44
infrastructure is actively verifying that no malicious workloads are running on the hardware. That's exactly what Stealthium does by catching these probes straight from the chrome behavior.
Thank you
Brzak closes by saying GPU timing can reveal the model, speed and workload even across NVLink. He credits the Behind Bars researchers whose work underpins the talk.
Text in image
Thank you. STEALTHIUM stealthium.io Branislav Brzak Security Researcher scan for stealthium.io
- 12:57
Sharing a GPU means timing alone can give away the model, the speed, and the workload. That's true even across NVLink without needing any exploit at all. Stealthium sees this happen in real time. Most organizations still don't have that kind of visibility. Thanks again to the researchers of Behind Bars. This whole talk builds upon their work. Thank you all for watching.