# Shipping like a startup, regulated like a bank

David Abitbol · AI Engineer NYC 2026 · October 13

[Watch recording](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20214s)

[05:36:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20215s) Hello everyone, welcome.

[05:36:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20219s) My name is David.

[05:37:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20221s) I lead AI operations at a company called Revolut.

[05:37:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20225s) We have 15 minutes today together.

[05:37:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20229s) we'll talk about high performance engineering and self-hosting open source model on dedicated GPU.

[05:37:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20235s) How does that sound?

[05:37:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20238s) Cool.

[05:37:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20239s) So it's going to be a technical one, so bear with me.

[05:37:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20242s) So maybe to start off, because I know this is the US and you may not know Revolut,

[05:37:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20248s) quick words about our company.

[05:37:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20251s) We're a digital bank in Europe.

[05:37:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20253s) We have 80 million customers.

[05:37:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20255s) We're live in 45 markets.

[05:37:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20258s) with the most downloaded financial app in Europe for the last year.

[05:37:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20263s) And as of July, we're Europe's largest private company.

[05:37:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20269s) So we're doing okay.

[05:37:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20273s) The emphasis of Revolut is shipping a lot of product,

[05:37:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20276s) and so there's a high focus on engineering culture and excellence on that demand.

[05:38:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20282s) And so we'll talk about that.

[05:38:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20285s) So let's talk about some AI.

[05:38:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20289s) We have a chart here.

[05:38:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20290s) I'm going to see two lines.

[05:38:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20292s) This is a sample of 2,000 engineers' performance over the last year.

[05:38:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20297s) So the green line is going to be lead time.

[05:38:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20300s) So it's going to be when an engineer picks up a feature

[05:38:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20303s) until a feature is merged and ready to be deployed.

[05:38:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20308s) The blue line is going to be coding time.

[05:38:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20310s) So the green line is in days.

[05:38:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20311s) The blue line is in hours.

[05:38:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20315s) Just to reset the timeline in case people forget,

[05:38:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20317s) because AI moves really fast.

[05:38:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20319s) Claude Code came out around February 2025, right?

[05:38:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20323s) And that's the Claude Code, Sonnet 3.7,

[05:38:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20326s) the one that would delete your prod database, right?

[05:38:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20329s) Not the safe cloud and codex that we have today.

[05:38:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20332s) So bear in mind when we talk about adoption.

[05:38:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20335s) So what you see on this chart is January 25,

[05:38:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20339s) first introduced AI in the company, right?

[05:39:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20341s) Gemini.

[05:39:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20342s) First time people were able to use private data within AI.

[05:39:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20348s) Around February, we started playing around with Claude Code, and then January is when

[05:39:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20352s) we first launched Claude Code at scale.

[05:39:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20356s) What you see is Gemini had 10% improvement in lead time, 20% improvement in coding time,

[05:39:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20362s) so that's people no longer need to look at Stack Overflow and reading documentation.

[05:39:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20367s) But then Claude Code, not so much impact at the time.

[05:39:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20371s) So when we launched it, we were highly skeptical about the cost of AI.

[05:39:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20376s) we were seeing engineers spend a thousand a month,

[05:39:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20379s) 10,000 a month with not so much ROI.

[05:39:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20381s) And so when we launched it,

[05:39:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20383s) we launched it with strict budget limits.

[05:39:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20387s) And so we're looking at the next thing, right?

[05:39:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20388s) So how can we provide limited tokens to our engineers

[05:39:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20391s) so they can use AI guild-free for a fraction of the cost?

[05:39:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20395s) And so we looked into open source.

[05:39:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20398s) So around April 26 is when we started

[05:40:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20400s) self-hosting open source model.

[05:40:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20402s) We're a big fan of GLM for coding.

[05:40:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20405s) so around that time what you see

[05:40:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20406s) April 26th, GLM 5.1

[05:40:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20410s) we see 60% improvement

[05:40:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20412s) in coding time

[05:40:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20414s) 70% improvement in lead time

[05:40:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20417s) so traumatic impact

[05:40:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20420s) what do we learn from this?

[05:40:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20421s) well there's more impact on the lead time

[05:40:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20422s) than on the coding time

[05:40:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20423s) so what we find is the bottleneck for engineering

[05:40:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20426s) is not necessarily coding

[05:40:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20429s) you can save a lot of time

[05:40:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20430s) not having to dig through your inbox

[05:40:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20432s) or messing with Jira and Confluence

[05:40:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20434s) and so when you have a limited token you're free to do that. You no longer have to save

[05:40:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20439s) your tokens for the end of the quarter or the most difficult task. So where do we stand?

[05:40:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20447s) Well open source is now a priority, a revolute, and your economics are favorable, the billing

[05:40:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20453s) is flat, we don't have any bill shock any of the month, and the frontier is the exception.

[05:40:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20459s) So obviously we keep using codex, code code.

[05:41:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20462s) The latest model are fantastic.

[05:41:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20464s) Jimmy and I are going very good as well.

[05:41:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20467s) But we keep them as exception.

[05:41:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20471s) So how do we get there?

[05:41:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20474s) Some metrics that matter when you're self-hosting model.

[05:41:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20478s) So to serve 2,000 engineers, we roughly need 56 B200.

[05:41:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20483s) And that's to host GLM 5.3.

[05:41:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20486s) GLM is a big model, right?

[05:41:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20488s) 8.500 billion parameters.

[05:41:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20491s) What you want to aim for is

[05:41:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20493s) sub-3 second time to first token.

[05:41:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20497s) This is empirical evidence.

[05:41:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20498s) As soon as you exceed 5 seconds,

[05:41:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20501s) my Slack inbox starts to blow up,

[05:41:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20503s) and so we know that's the limit.

[05:41:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20506s) In terms of throughput,

[05:41:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20507s) it doesn't really matter as much.

[05:41:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20508s) We serve, in terms of interactivity,

[05:41:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20511s) 200 tokens per second.

[05:41:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20512s) People are fine.

[05:41:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20513s) You don't need to go higher.

[05:41:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20515s) The way people work has changed.

[05:41:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20516s) which we no longer look at the stream of thinking tokens.

[05:42:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20520s) People run tasks in parallel

[05:42:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20521s) and check in when the task is done.

[05:42:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20524s) But if you're in a high throughput

[05:42:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20525s) and you're an engineer at Revolut,

[05:42:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20527s) you just connect after 5 p.m. when one is on GPU

[05:42:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20529s) you can get 800 TPS and then, you know, it's a dream.

[05:42:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20535s) In terms of cost, in terms of impact,

[05:42:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20537s) self-hosting, GLM, or blended cost of token

[05:42:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20539s) is 30 cents per million.

[05:42:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20542s) And so our invoice is one-tenth

[05:42:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20544s) of what we were spending on Anthropic and OpenAI.

[05:42:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20552s) So there's a lot of tuning

[05:42:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20554s) that goes into running open source models.

[05:42:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20558s) If you just get GPUs and upload weights on a GPU

[05:42:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20563s) and you start doing some inference,

[05:42:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20566s) you're going to get terrible performance.

[05:42:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20567s) That's because your workload is going to be very dependent

[05:42:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20571s) on the parameters that you set.

[05:42:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20574s) So if you look all the way to the left of that chart, the blue line is going to be demand,

[05:42:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20578s) so that's request per second.

[05:42:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20579s) That's going to be what's going to be the bottleneck for your latency for your GPUs.

[05:43:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20584s) And the red line is what we're optimizing for, that's the time to first token.

[05:43:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20588s) So when we started back in April, you could see request per second around 3,

[05:43:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20595s) which is high but not so high, and time to first token over 10 seconds, which is very slow.

[05:43:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20601s) All of the data that you see is

[05:43:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20603s) the same number of GPU. We did an increase GPU count with LLF56.

[05:43:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20608s) And by the end, you see in August, requests per second were over 7, and time to first occur, and we're around 1.6.

[05:43:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20615s) So we've had 88% improvement in performance for 76% increased demand.

[05:43:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20623s) So the learning from this is there's a lot of parameters that you can improve on, and you really need to resist scaling.

[05:43:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20628s) if you're looking to self-host a model.

[05:43:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20632s) So let's talk parameters.

[05:43:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20635s) There's no secret recipe,

[05:43:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20636s) but the things you consider are the following.

[05:44:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20641s) First one, long context is a scam.

[05:44:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20644s) You really need to be mindful that

[05:44:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20646s) even though you can serve one million token,

[05:44:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20649s) 256 is really what you need.

[05:44:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20653s) Quantization, if you're running Blackwell generation GPU,

[05:44:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20656s) you don't want to run anything higher than FP4.

[05:44:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20659s) And the FP4 is best, roughly going from FP4 to FP8

[05:44:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20664s) is a 40% saving in GPU footprint for zero loss in precision.

[05:44:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20671s) Other parameters to consider, tensor parallelism,

[05:44:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20674s) even though GLM fits on TP4, you don't have enough KV

[05:44:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20678s) cache, and so maybe a TP8 works better,

[05:44:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20680s) depending on how large your workload is.

[05:44:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20683s) and then on the more esoteric

[05:44:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20685s) check your cache hit

[05:44:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20686s) coding test needs to be above 90%

[05:44:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20689s) cache hit

[05:44:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20691s) and you can look into optimize

[05:44:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20693s) spec decoding

[05:44:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20694s) sequential pre-fill, PD disaggregation

[05:44:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20697s) all of these

[05:45:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20700s) cool

[05:45:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20701s) so now let's talk about production setup

[05:45:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20704s) GLM obviously

[05:45:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20705s) is a text only model

[05:45:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20707s) which is a limitation for some people

[05:45:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20709s) the trade off I would say

[05:45:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20711s) we would make every time is get the smarter model and figure out how to do

[05:45:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20714s) image recognition another way. You can easily have a local router, open codex is

[05:45:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20720s) one, where you would route any image recognition to Gemini or Luna and keep

[05:45:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20726s) GLM for the main task. If you really need some vision model, GLM 5G flash works

[05:45:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20734s) really well. We have our UI designer making great use of it. Turnaround for UI product has gone a

[05:45:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20744s) really, really long way. And then the last one is going to be about the harness. So all of these

[05:45:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20752s) endpoints are going to be OpenAI compatible API with response API. And so you can use them

[05:45:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20759s) inside OpenCode Codex, Codex app as well, works very well.

[05:46:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20763s) And so that's what we recommend.

[05:46:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20770s) And then last message, this one is for the CIO in the room.

[05:46:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20776s) There's been a lot of stigma in using Chinese open source model.

[05:46:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20781s) So the thing to consider is you're not using

[05:46:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20784s) inference, dedicated endpoint that goes to China.

[05:46:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20787s) you're just getting the weights and putting them on a US-based service.

[05:46:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20792s) So for data sharing side, this is totally fine.

[05:46:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20797s) Revolut in particular, because we're an EU-regulated bank,

[05:46:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20801s) we fall under the most strict regime in terms of compliance.

[05:46:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20806s) so what we find that works is all of these coding tasks are not considered autonomous agents.

[05:46:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20813s) They're always with a human in the loop.

[05:46:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20815s) And so how do you give reassurance

[05:46:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20818s) that you have a human loop?

[05:47:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20820s) You start by creating a sandbox that guarantees that.

[05:47:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20824s) And the sandbox means now if you've used Codex Claude Code,

[05:47:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20827s) it has all of these features of approve for me.

[05:47:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20830s) You just have to set the rules for the operations

[05:47:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20832s) that you don't want to allow.

[05:47:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20833s) So you can allow grep but not gradle, for instance.

[05:47:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20837s) And that's gonna give confidence to your regulator

[05:47:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20839s) that it's not an autonomous agent

[05:47:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20842s) that needs different supervision.

[05:47:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20846s) This sandbox also protects you from rogue use.

[05:47:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20849s) You may have bad actors inside your company, and you want to shield for that.

[05:47:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20853s) You'd be surprised what you find in a large company.

[05:47:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20857s) And then the last one is prompt injection.

[05:47:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20859s) Not so much a thing now, given how smart models are, but always a thing.

[05:47:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20865s) In terms of data privacy, we spoke about it.

[05:47:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20867s) As long as the data sits in one of the countries that are in the data equivalency framework,

[05:47:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20872s) so US, EU, UK, Israel, you're fine, you can get your GPUs there. All of your inference provider

[05:48:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20880s) or GPU provider are going to have zero data retention by default. And then the last thing

[05:48:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20887s) you need is a very strong open telemetry so that if you ever have an incident, you need to be able

[05:48:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20894s) to go deep and look at the logs.

[05:48:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20898s) Also gives you some analytics.

[05:48:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20904s) All right, thanks everyone.

[05:48:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20905s) That was all for my talk.