[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12720,
    "text": ">> Hello everyone."
  },
  {
    "startMs": 14440,
    "text": "So, preferences over benchmarks."
  },
  {
    "startMs": 17480,
    "text": "The talk today is about model routing"
  },
  {
    "startMs": 20600,
    "text": "and specifically why the way most people"
  },
  {
    "startMs": 23000,
    "text": "think about picking a model, which"
  },
  {
    "startMs": 25120,
    "text": "usually is chasing, you know, to the top"
  },
  {
    "startMs": 26920,
    "text": "of a benchmark, is actually the wrong"
  },
  {
    "startMs": 29160,
    "text": "instinct."
  },
  {
    "startMs": 32119,
    "text": "I'm Archana, VP of engineering for"
  },
  {
    "startMs": 34200,
    "text": "inference engine and AI infrastructure"
  },
  {
    "startMs": 36440,
    "text": "at DigitalOcean. And I'll be joined by"
  },
  {
    "startMs": 38800,
    "text": "Tyler, who built parts of the router and"
  },
  {
    "startMs": 41280,
    "text": "will actually do a live demo for us"
  },
  {
    "startMs": 42840,
    "text": "today."
  },
  {
    "startMs": 44000,
    "text": "We both work on the managed agent"
  },
  {
    "startMs": 45680,
    "text": "orchestration and inference engine"
  },
  {
    "startMs": 47440,
    "text": "products at DigitalOcean."
  },
  {
    "startMs": 52200,
    "text": "So, you may know DigitalOcean as"
  },
  {
    "startMs": 54000,
    "text": "droplets, databases, and app platform."
  },
  {
    "startMs": 57520,
    "text": "All of that is true. We are also the AI"
  },
  {
    "startMs": 59920,
    "text": "native cloud."
  },
  {
    "startMs": 61680,
    "text": "This is five integrated layers, starting"
  },
  {
    "startMs": 64400,
    "text": "from infrastructure all the way up to"
  },
  {
    "startMs": 66160,
    "text": "the managed agents, with the inference"
  },
  {
    "startMs": 68480,
    "text": "engine right in the middle."
  },
  {
    "startMs": 70560,
    "text": "And that's why we are here talking about"
  },
  {
    "startMs": 72280,
    "text": "inference router. The routing lives in"
  },
  {
    "startMs": 74680,
    "text": "the inference engine, and if you want to"
  },
  {
    "startMs": 76400,
    "text": "know more about our stack and the full"
  },
  {
    "startMs": 77880,
    "text": "story, please come find us at the booth."
  },
  {
    "startMs": 84000,
    "text": "So, everybody is reaching out for the"
  },
  {
    "startMs": 85920,
    "text": "model routing. And let's look at three"
  },
  {
    "startMs": 88240,
    "text": "reasons why the three reasons that are"
  },
  {
    "startMs": 90440,
    "text": "breaking the one model habit for most"
  },
  {
    "startMs": 92560,
    "text": "users."
  },
  {
    "startMs": 93840,
    "text": "The first one I want to talk about is"
  },
  {
    "startMs": 95120,
    "text": "cost."
  },
  {
    "startMs": 96520,
    "text": "Spend is exploding, and even companies"
  },
  {
    "startMs": 98680,
    "text": "like Walmart, Uber, Microsoft, they're"
  },
  {
    "startMs": 101600,
    "text": "actively capping usage to control the"
  },
  {
    "startMs": 103840,
    "text": "inference bills."
  },
  {
    "startMs": 106120,
    "text": "Second one is fit."
  },
  {
    "startMs": 108120,
    "text": "One model for every task is likely an"
  },
  {
    "startMs": 110360,
    "text": "overkill."
  },
  {
    "startMs": 111600,
    "text": "You're essentially paying frontier rates"
  },
  {
    "startMs": 113720,
    "text": "for a work that a much smaller model"
  },
  {
    "startMs": 115600,
    "text": "will be able to handle really well."
  },
  {
    "startMs": 118280,
    "text": "And the third one, which for me is the"
  },
  {
    "startMs": 119760,
    "text": "most important one, is the risk."
  },
  {
    "startMs": 122520,
    "text": "The risk associated with one single"
  },
  {
    "startMs": 124520,
    "text": "model."
  },
  {
    "startMs": 125640,
    "text": "Models can go down, and if you bet your"
  },
  {
    "startMs": 127800,
    "text": "entire product and production on one"
  },
  {
    "startMs": 130160,
    "text": "model, you have no failover when"
  },
  {
    "startMs": 132200,
    "text": "something degrades."
  },
  {
    "startMs": 136360,
    "text": "And model orchestration is actually the"
  },
  {
    "startMs": 138400,
    "text": "new FinOps."
  },
  {
    "startMs": 140760,
    "text": "As you all know, cloud cost optimization"
  },
  {
    "startMs": 143440,
    "text": "took us about 15 years for it to"
  },
  {
    "startMs": 145880,
    "text": "actually become a real good discipline"
  },
  {
    "startMs": 147720,
    "text": "and for companies to get it right."
  },
  {
    "startMs": 149840,
    "text": "This one actually is arriving in months"
  },
  {
    "startMs": 151320,
    "text": "and not years. And here's the premise"
  },
  {
    "startMs": 153800,
    "text": "that I think everybody gets wrong about"
  },
  {
    "startMs": 155440,
    "text": "this."
  },
  {
    "startMs": 157959,
    "text": "We all think of like what is the best"
  },
  {
    "startMs": 159680,
    "text": "model for a job. Here's the thing. There"
  },
  {
    "startMs": 162600,
    "text": "is no single best model."
  },
  {
    "startMs": 165160,
    "text": "The right one depends on the actual"
  },
  {
    "startMs": 167080,
    "text": "request."
  },
  {
    "startMs": 168440,
    "text": "For example, if you're doing"
  },
  {
    "startMs": 170200,
    "text": "classification and labeling,"
  },
  {
    "startMs": 172440,
    "text": "a small open model may very well work"
  },
  {
    "startMs": 174760,
    "text": "really well for you and will give you"
  },
  {
    "startMs": 176400,
    "text": "really good cost optimizations."
  },
  {
    "startMs": 178959,
    "text": "However, if you're running code"
  },
  {
    "startMs": 180959,
    "text": "completion in line, you will likely need"
  },
  {
    "startMs": 183600,
    "text": "really fast routing, and that is where a"
  },
  {
    "startMs": 185680,
    "text": "faster, larger routing model comes into"
  },
  {
    "startMs": 188040,
    "text": "picture."
  },
  {
    "startMs": 189640,
    "text": "Think about code generation and bug"
  },
  {
    "startMs": 191080,
    "text": "fixing. You're likely good with an mid"
  },
  {
    "startMs": 193800,
    "text": "open weight model, uh and again, it'll"
  },
  {
    "startMs": 196239,
    "text": "bring you like really good cost"
  },
  {
    "startMs": 197320,
    "text": "optimizations over using a frontier for"
  },
  {
    "startMs": 199560,
    "text": "something that is likely an overkill in"
  },
  {
    "startMs": 201519,
    "text": "this situation."
  },
  {
    "startMs": 203200,
    "text": "But then you're looking at like really"
  },
  {
    "startMs": 204680,
    "text": "accuracy-critical tasks, like code"
  },
  {
    "startMs": 207040,
    "text": "review and security, you're likely going"
  },
  {
    "startMs": 209560,
    "text": "to lean towards a frontier model."
  },
  {
    "startMs": 213959,
    "text": "So, essentially, what makes a model"
  },
  {
    "startMs": 215959,
    "text": "right for a request?"
  },
  {
    "startMs": 217959,
    "text": "It's a mix that no public leaderboard"
  },
  {
    "startMs": 220000,
    "text": "can actually encode for you."
  },
  {
    "startMs": 222400,
    "text": "Because it's the task itself. What are"
  },
  {
    "startMs": 224120,
    "text": "you actually trying to achieve? What is"
  },
  {
    "startMs": 225600,
    "text": "your model trying to achieve?"
  },
  {
    "startMs": 227560,
    "text": "The system prompts and tools around it,"
  },
  {
    "startMs": 229440,
    "text": "that is the methodology by which you're"
  },
  {
    "startMs": 231560,
    "text": "getting something done using a model."
  },
  {
    "startMs": 234280,
    "text": "The cost you're willing to spend, this"
  },
  {
    "startMs": 235600,
    "text": "is a very, very important aspect."
  },
  {
    "startMs": 238560,
    "text": "And latency the use case needs, not all"
  },
  {
    "startMs": 241120,
    "text": "use cases need the same amount of"
  },
  {
    "startMs": 243120,
    "text": "latency. So, depending on what you're"
  },
  {
    "startMs": 244800,
    "text": "trying to do, this can vary widely. And"
  },
  {
    "startMs": 247040,
    "text": "finally, the end user preference. All of"
  },
  {
    "startMs": 248560,
    "text": "this is driven by what the end user"
  },
  {
    "startMs": 250440,
    "text": "really wants out of your application."
  },
  {
    "startMs": 253640,
    "text": "So, I'm going to welcome Tyler onto"
  },
  {
    "startMs": 255680,
    "text": "stage so that he can actually show you"
  },
  {
    "startMs": 257320,
    "text": "the Inference Light Router live in"
  },
  {
    "startMs": 259079,
    "text": "action and show you how it can really"
  },
  {
    "startMs": 261600,
    "text": "help with all of these key aspects that"
  },
  {
    "startMs": 264040,
    "text": "I'm calling out here."
  },
  {
    "startMs": 271280,
    "text": ">> Testing. All right. Thank you, Archana."
  },
  {
    "startMs": 275720,
    "text": "Okay. So, many builders"
  },
  {
    "startMs": 278480,
    "text": "have tried auto routing before."
  },
  {
    "startMs": 280640,
    "text": "But the problem was that it feels like a"
  },
  {
    "startMs": 282480,
    "text": "black box."
  },
  {
    "startMs": 284160,
    "text": "The router makes a choice, and if that"
  },
  {
    "startMs": 285960,
    "text": "choice results in poor performance, you"
  },
  {
    "startMs": 287880,
    "text": "really have no way of improving it."
  },
  {
    "startMs": 289800,
    "text": "We built ours differently."
  },
  {
    "startMs": 291640,
    "text": "At the architecture level, which is what"
  },
  {
    "startMs": 293120,
    "text": "you can see on the screen,"
  },
  {
    "startMs": 294520,
    "text": "a request runs through our open proxy"
  },
  {
    "startMs": 296440,
    "text": "planner"
  },
  {
    "startMs": 297520,
    "text": "and our purpose-built routing model,"
  },
  {
    "startMs": 299600,
    "text": "both open source."
  },
  {
    "startMs": 301600,
    "text": "There's no vendor lock-in, which is a"
  },
  {
    "startMs": 303160,
    "text": "key DigitalOcean value."
  },
  {
    "startMs": 306680,
    "text": "You describe what matters for your"
  },
  {
    "startMs": 307800,
    "text": "workflow,"
  },
  {
    "startMs": 309040,
    "text": "costs, latency, quality, preferred"
  },
  {
    "startMs": 312080,
    "text": "models,"
  },
  {
    "startMs": 314880,
    "text": "or hardware. Then the router uses that"
  },
  {
    "startMs": 316960,
    "text": "context to pick the right model per"
  },
  {
    "startMs": 319160,
    "text": "request."
  },
  {
    "startMs": 320720,
    "text": "Because the routing model is specialized"
  },
  {
    "startMs": 322160,
    "text": "for this job, it's super fast, under 200"
  },
  {
    "startMs": 324720,
    "text": "milliseconds, and it costs customers"
  },
  {
    "startMs": 326640,
    "text": "nothing extra. In our evaluations, it"
  },
  {
    "startMs": 329360,
    "text": "actually has beaten frontier models like"
  },
  {
    "startMs": 330880,
    "text": "the GPT-5 series models at routing tasks"
  },
  {
    "startMs": 333760,
    "text": "itself with a fraction of the latency."
  },
  {
    "startMs": 337440,
    "text": "So, the difference is simple."
  },
  {
    "startMs": 339040,
    "text": "This is routing you can customize,"
  },
  {
    "startMs": 340800,
    "text": "evaluate, and improve without vendor"
  },
  {
    "startMs": 343560,
    "text": "lock-in."
  },
  {
    "startMs": 348280,
    "text": "So, you bring your preferences and we"
  },
  {
    "startMs": 350040,
    "text": "honor them. You describe a task in"
  },
  {
    "startMs": 352160,
    "text": "natural language and set what matters,"
  },
  {
    "startMs": 354600,
    "text": "cost, latency, and task description. You"
  },
  {
    "startMs": 357880,
    "text": "bring your rules and we execute them"
  },
  {
    "startMs": 359840,
    "text": "intelligently."
  },
  {
    "startMs": 361400,
    "text": "Layer decision tree rules on top, start"
  },
  {
    "startMs": 363760,
    "text": "from presets, change anything you want"
  },
  {
    "startMs": 366280,
    "text": "in a single line of code."
  },
  {
    "startMs": 368040,
    "text": "And you validate with your own"
  },
  {
    "startMs": 369120,
    "text": "evaluations, not someone else's"
  },
  {
    "startMs": 370760,
    "text": "leaderboard. Route, evaluate, adjust,"
  },
  {
    "startMs": 375120,
    "text": "then feed that back in. That loop is"
  },
  {
    "startMs": 377400,
    "text": "key."
  },
  {
    "startMs": 379960,
    "text": "Okay, we're going to switch gears here."
  },
  {
    "startMs": 381360,
    "text": "We're going to do a live demo."
  },
  {
    "startMs": 385760,
    "text": "Bear with me here."
  },
  {
    "startMs": 389520,
    "text": "All right, I'm going to show you a"
  },
  {
    "startMs": 390160,
    "text": "couple of things. First, I'll show you"
  },
  {
    "startMs": 391919,
    "text": "router configuration in the UI, how to"
  },
  {
    "startMs": 394080,
    "text": "use it, and then how you can use"
  },
  {
    "startMs": 395680,
    "text": "evaluations to measure and improve your"
  },
  {
    "startMs": 398360,
    "text": "router's performance."
  },
  {
    "startMs": 399800,
    "text": "And then I'll show you a real router"
  },
  {
    "startMs": 401680,
    "text": "that I created inside a coding agent"
  },
  {
    "startMs": 403960,
    "text": "workflow."
  },
  {
    "startMs": 405800,
    "text": "So I'm here in the cloud console, the"
  },
  {
    "startMs": 407080,
    "text": "DigitalOcean cloud console. And you can"
  },
  {
    "startMs": 409000,
    "text": "see my routers. We have several presets."
  },
  {
    "startMs": 411520,
    "text": "You can see software engineering in"
  },
  {
    "startMs": 412720,
    "text": "general writing,"
  },
  {
    "startMs": 414320,
    "text": "knowledge bases and document"
  },
  {
    "startMs": 415320,
    "text": "intelligence. In this case, I've"
  },
  {
    "startMs": 417480,
    "text": "actually created my own. So I I"
  },
  {
    "startMs": 419080,
    "text": "customized our preset software"
  },
  {
    "startMs": 420680,
    "text": "engineering. Uh if we click into this,"
  },
  {
    "startMs": 423480,
    "text": "we can see that I have several several"
  },
  {
    "startMs": 425280,
    "text": "different tasks here."
  },
  {
    "startMs": 426880,
    "text": "I have bug fixing, code generation, test"
  },
  {
    "startMs": 429280,
    "text": "writing, and a few others."
  },
  {
    "startMs": 431240,
    "text": "This also shows that you can specify"
  },
  {
    "startMs": 433520,
    "text": "more than one model per task"
  },
  {
    "startMs": 435760,
    "text": "in the bug fixing case and code"
  },
  {
    "startMs": 436880,
    "text": "generation case."
  },
  {
    "startMs": 438360,
    "text": "Um in the code generation, I have GLM"
  },
  {
    "startMs": 440000,
    "text": "5.2 and GPT 5.2."
  },
  {
    "startMs": 442960,
    "text": "And because I really want to always"
  },
  {
    "startMs": 444200,
    "text": "route to GLM 5.2 unless it's down, I use"
  },
  {
    "startMs": 447200,
    "text": "this manual ranking option. So it'll"
  },
  {
    "startMs": 448919,
    "text": "always go to GLM 5.2. If GLM fails,"
  },
  {
    "startMs": 451919,
    "text": "it'll fail over to GPT 5.2."
  },
  {
    "startMs": 455200,
    "text": "In the bug fixing one, you can see a"
  },
  {
    "startMs": 456440,
    "text": "little bit of a different one. In this"
  },
  {
    "startMs": 457919,
    "text": "case, I have selection policy fastest."
  },
  {
    "startMs": 460919,
    "text": "So out of this model pool, if it matches"
  },
  {
    "startMs": 463080,
    "text": "to bug fixing, it'll pick whichever"
  },
  {
    "startMs": 464640,
    "text": "one's been fastest in about the last 30"
  },
  {
    "startMs": 466680,
    "text": "minutes."
  },
  {
    "startMs": 468840,
    "text": "Okay, let's do this in action a little"
  },
  {
    "startMs": 469880,
    "text": "bit. Here's our playground. We're all"
  },
  {
    "startMs": 471800,
    "text": "show a couple of examples side by side."
  },
  {
    "startMs": 474800,
    "text": "First, I'll start with this is a simple"
  },
  {
    "startMs": 476400,
    "text": "prompt, write a basic Fibonacci"
  },
  {
    "startMs": 477840,
    "text": "function."
  },
  {
    "startMs": 481040,
    "text": "And as this runs, we can see on the left"
  },
  {
    "startMs": 483560,
    "text": "we're writing to Opus."
  },
  {
    "startMs": 485160,
    "text": "On the right we're using our software"
  },
  {
    "startMs": 486720,
    "text": "engineering router that I just showed"
  },
  {
    "startMs": 487880,
    "text": "you."
  },
  {
    "startMs": 488880,
    "text": "And you're going to see that it picks"
  },
  {
    "startMs": 489800,
    "text": "different models on the right. So, in"
  },
  {
    "startMs": 491200,
    "text": "this case, it matched to the code"
  },
  {
    "startMs": 492960,
    "text": "snippets task and just used the llama"
  },
  {
    "startMs": 495000,
    "text": "format router model that I had"
  },
  {
    "startMs": 496560,
    "text": "configured for that one. And if we"
  },
  {
    "startMs": 498120,
    "text": "scroll down, I mean this is this is"
  },
  {
    "startMs": 499640,
    "text": "obvious, right? But this model is"
  },
  {
    "startMs": 501320,
    "text": "extremely fast and extremely cheap"
  },
  {
    "startMs": 503360,
    "text": "compared to Opus."
  },
  {
    "startMs": 507640,
    "text": "Now, let's say optimize my function."
  },
  {
    "startMs": 510960,
    "text": "And we'll see the same thing happen. In"
  },
  {
    "startMs": 512280,
    "text": "this case, it matched to the code"
  },
  {
    "startMs": 513360,
    "text": "performance optimization task using"
  },
  {
    "startMs": 515200,
    "text": "GPT-5.2."
  },
  {
    "startMs": 516719,
    "text": "And again, it's obviously significantly"
  },
  {
    "startMs": 518919,
    "text": "faster. If we scroll down here, we can"
  },
  {
    "startMs": 520719,
    "text": "also see that it's significantly"
  },
  {
    "startMs": 522520,
    "text": "cheaper."
  },
  {
    "startMs": 524000,
    "text": "We'll do one more, write some unit"
  },
  {
    "startMs": 525320,
    "text": "tests."
  },
  {
    "startMs": 530560,
    "text": "Okay, and in this case it matched to"
  },
  {
    "startMs": 532200,
    "text": "Claude 5 Summit on the test writing and"
  },
  {
    "startMs": 533920,
    "text": "code verification."
  },
  {
    "startMs": 536040,
    "text": "And again, we're going to see faster and"
  },
  {
    "startMs": 537880,
    "text": "cheaper."
  },
  {
    "startMs": 538880,
    "text": "So, it's a pattern. It matches my, you"
  },
  {
    "startMs": 540800,
    "text": "know, vibe check, right? It still vibes"
  },
  {
    "startMs": 542960,
    "text": "though. How you actually prove it is"
  },
  {
    "startMs": 544839,
    "text": "working it through evaluations."
  },
  {
    "startMs": 547720,
    "text": "So, I have an evaluation that I ran"
  },
  {
    "startMs": 549160,
    "text": "here."
  },
  {
    "startMs": 550560,
    "text": "Comparing Opus on the left or actually"
  },
  {
    "startMs": 553440,
    "text": "on the right hand side to my router on"
  },
  {
    "startMs": 555440,
    "text": "the left hand side."
  },
  {
    "startMs": 557240,
    "text": "You can see that the scores, 90% for my"
  },
  {
    "startMs": 559320,
    "text": "router, 95% correctness for Opus, are"
  },
  {
    "startMs": 562000,
    "text": "very, very close. In fact, that's pretty"
  },
  {
    "startMs": 564160,
    "text": "much within oh, it was a judge uh margin"
  },
  {
    "startMs": 567680,
    "text": "of error."
  },
  {
    "startMs": 569160,
    "text": "But we what's really interesting is if"
  },
  {
    "startMs": 570760,
    "text": "we scroll down here,"
  },
  {
    "startMs": 572200,
    "text": "we can see that the router used"
  },
  {
    "startMs": 574560,
    "text": "significantly less tokens and was"
  },
  {
    "startMs": 577120,
    "text": "significantly faster than Opus."
  },
  {
    "startMs": 581800,
    "text": "Okay, let's jump into a real workflow"
  },
  {
    "startMs": 583880,
    "text": "here. This is where the inference router"
  },
  {
    "startMs": 585400,
    "text": "really becomes impactful."
  },
  {
    "startMs": 587600,
    "text": "Here I have two terminals running open"
  },
  {
    "startMs": 589080,
    "text": "code."
  },
  {
    "startMs": 590040,
    "text": "On the left, I have a single model"
  },
  {
    "startMs": 591760,
    "text": "approach using quad Opus. So, I have"
  },
  {
    "startMs": 593600,
    "text": "Opus set up or open code set up with"
  },
  {
    "startMs": 595560,
    "text": "Opus."
  },
  {
    "startMs": 596560,
    "text": "On the right, I've configured open code"
  },
  {
    "startMs": 598000,
    "text": "to send requests to our software"
  },
  {
    "startMs": 599120,
    "text": "engineering router"
  },
  {
    "startMs": 600600,
    "text": "that I just showed you configure."
  },
  {
    "startMs": 603280,
    "text": "Um below, I kind of have this custom"
  },
  {
    "startMs": 604840,
    "text": "built open code where you'll be able to"
  },
  {
    "startMs": 606240,
    "text": "see live uh"
  },
  {
    "startMs": 608000,
    "text": "observability essentially."
  },
  {
    "startMs": 609760,
    "text": "So, let's go go ahead and get these"
  },
  {
    "startMs": 610680,
    "text": "started. It's just a simple"
  },
  {
    "startMs": 612560,
    "text": "feature request preloaded into here."
  },
  {
    "startMs": 614240,
    "text": "Build me a spinning wheel app."
  },
  {
    "startMs": 616120,
    "text": "I'll run the same prompt in both."
  },
  {
    "startMs": 618920,
    "text": "And as this runs, we can focus on the"
  },
  {
    "startMs": 620320,
    "text": "bottom panel. So, it'll start to show up"
  },
  {
    "startMs": 622080,
    "text": "here. Hopefully, you can see that on the"
  },
  {
    "startMs": 623200,
    "text": "screen."
  },
  {
    "startMs": 624480,
    "text": "Uh you'll be able to see token usage in"
  },
  {
    "startMs": 625760,
    "text": "real time, which models are being"
  },
  {
    "startMs": 627640,
    "text": "selected,"
  },
  {
    "startMs": 628840,
    "text": "what task those map to, and the cost"
  },
  {
    "startMs": 630960,
    "text": "accumulating live."
  },
  {
    "startMs": 632560,
    "text": "So, on the right, we can already see"
  },
  {
    "startMs": 633920,
    "text": "that we're starting to route to Gemini"
  },
  {
    "startMs": 635360,
    "text": "5.2 because our requests are starting to"
  },
  {
    "startMs": 638520,
    "text": "match the code generation."
  },
  {
    "startMs": 640280,
    "text": "And on the left, of course, we're just"
  },
  {
    "startMs": 641360,
    "text": "routing to quad Opus. I think open code"
  },
  {
    "startMs": 643720,
    "text": "sometimes routes to to Haiku by itself."
  },
  {
    "startMs": 645600,
    "text": "So, that's what you see there."
  },
  {
    "startMs": 648040,
    "text": "And we'll notice the latency, too, how"
  },
  {
    "startMs": 649400,
    "text": "quickly things start to come back. In"
  },
  {
    "startMs": 651080,
    "text": "this case, it wants me to"
  },
  {
    "startMs": 654440,
    "text": "create a temporary directory."
  },
  {
    "startMs": 658600,
    "text": "So, the key difference here is that on"
  },
  {
    "startMs": 659920,
    "text": "the left, we'll see every single request"
  },
  {
    "startMs": 661640,
    "text": "that I write goes to the same premium"
  },
  {
    "startMs": 663400,
    "text": "model. Cost and latency are going to"
  },
  {
    "startMs": 665200,
    "text": "stay high for pretty much every single"
  },
  {
    "startMs": 666760,
    "text": "task. On the right, the router is"
  },
  {
    "startMs": 669280,
    "text": "selecting models based on the task."
  },
  {
    "startMs": 673347,
    "text": ">> [snorts]"
  },
  {
    "startMs": 673400,
    "text": ">> So, we're optimizing both cost and"
  },
  {
    "startMs": 675040,
    "text": "speed."
  },
  {
    "startMs": 676400,
    "text": "And we can see that uh our software"
  },
  {
    "startMs": 677839,
    "text": "engineering router already finished. And"
  },
  {
    "startMs": 679720,
    "text": "if we look here, it actually matched to"
  },
  {
    "startMs": 681200,
    "text": "two models throughout. So, let's go"
  },
  {
    "startMs": 682640,
    "text": "ahead and open this app and see how it"
  },
  {
    "startMs": 683640,
    "text": "looks."
  },
  {
    "startMs": 685040,
    "text": "Okay, this actually looks really solid"
  },
  {
    "startMs": 686960,
    "text": "to me. And Opus 4.7 finished at a"
  },
  {
    "startMs": 689640,
    "text": "similar time. Let's take a look at that."
  },
  {
    "startMs": 692400,
    "text": "We can compare them. I mean, it's This"
  },
  {
    "startMs": 693640,
    "text": "is a vibe check, right? But honestly, I"
  },
  {
    "startMs": 695400,
    "text": "would say the software engineer router"
  },
  {
    "startMs": 696640,
    "text": "did better cuz this is an interesting"
  },
  {
    "startMs": 698000,
    "text": "approach that you"
  },
  {
    "startMs": 699880,
    "text": "I'm not even sure it works too well. So,"
  },
  {
    "startMs": 701440,
    "text": "in this case,"
  },
  {
    "startMs": 702680,
    "text": "the router did a little bit better."
  },
  {
    "startMs": 705280,
    "text": "So, now that last step is done, you"
  },
  {
    "startMs": 706480,
    "text": "know, we get similar outputs, but if we"
  },
  {
    "startMs": 708400,
    "text": "look here,"
  },
  {
    "startMs": 710600,
    "text": "the software engineer router has only"
  },
  {
    "startMs": 712160,
    "text": "spent 8 cents on the session,"
  },
  {
    "startMs": 714560,
    "text": "while Opus directly has spent 25 cents."
  },
  {
    "startMs": 716760,
    "text": "So, we have a about a 3x in cost and"
  },
  {
    "startMs": 719200,
    "text": "very, very similar quality so far."
  },
  {
    "startMs": 723720,
    "text": "Let's try another another prompt here."
  },
  {
    "startMs": 725520,
    "text": "What What comes next in a software"
  },
  {
    "startMs": 726680,
    "text": "engineering life cycle? Probably writing"
  },
  {
    "startMs": 728080,
    "text": "some unit tests, right?"
  },
  {
    "startMs": 732200,
    "text": "So, we'll write this in both."
  },
  {
    "startMs": 736080,
    "text": "Start with this first. On"
  },
  {
    "startMs": 737540,
    "text": "[clears throat] the right, we have the"
  },
  {
    "startMs": 739920,
    "text": "router again. And we can see that it got"
  },
  {
    "startMs": 741800,
    "text": "matched to the test writing and code"
  },
  {
    "startMs": 743520,
    "text": "verification, which picked the Claude 5"
  },
  {
    "startMs": 745760,
    "text": "Sonnet"
  },
  {
    "startMs": 747040,
    "text": "model because that's what I configured"
  },
  {
    "startMs": 748320,
    "text": "earlier."
  },
  {
    "startMs": 749400,
    "text": "And we'll see the same pattern. It's"
  },
  {
    "startMs": 750600,
    "text": "going to be significantly cheaper"
  },
  {
    "startMs": 752360,
    "text": "overall across the entire session than"
  },
  {
    "startMs": 754800,
    "text": "going straight to Opus."
  },
  {
    "startMs": 757480,
    "text": "So, we'll let this finish here."
  },
  {
    "startMs": 762880,
    "text": "Okay, and that one finished. Let's just"
  },
  {
    "startMs": 764800,
    "text": "queue up one more. Write some"
  },
  {
    "startMs": 766640,
    "text": "documentation in a read me."
  },
  {
    "startMs": 774320,
    "text": "And then we'll compare"
  },
  {
    "startMs": 776160,
    "text": "the total session cost."
  },
  {
    "startMs": 783520,
    "text": "Okay, and as this runs, we'll wait and"
  },
  {
    "startMs": 785800,
    "text": "see what it does."
  },
  {
    "startMs": 789600,
    "text": "Okay, it created the read me. And well,"
  },
  {
    "startMs": 791160,
    "text": "if we look here, we can see that the"
  },
  {
    "startMs": 793080,
    "text": "total session cost"
  },
  {
    "startMs": 794920,
    "text": "for the router was 14 cents, while the"
  },
  {
    "startMs": 797200,
    "text": "total session cost for Opus was 44"
  },
  {
    "startMs": 799520,
    "text": "cents."
  },
  {
    "startMs": 801040,
    "text": "So, at this point, we can see the cost"
  },
  {
    "startMs": 802680,
    "text": "is significantly lower,"
  },
  {
    "startMs": 804360,
    "text": "latency is optimized per step,"
  },
  {
    "startMs": 806920,
    "text": "and the quality remains pretty similar"
  },
  {
    "startMs": 808560,
    "text": "across. So, you can see as you scale"
  },
  {
    "startMs": 810520,
    "text": "this, the cost"
  },
  {
    "startMs": 812800,
    "text": "performance really add up."
  },
  {
    "startMs": 815200,
    "text": "Okay. Archana, thank you."
  },
  {
    "startMs": 821379,
    "text": ">> [applause]"
  },
  {
    "startMs": 824800,
    "text": ">> Thank you so much, Tyler. And that was"
  },
  {
    "startMs": 827200,
    "text": "actually a live demo that we ran here."
  },
  {
    "startMs": 829560,
    "text": "So, thanks to Tyler for setting it up"
  },
  {
    "startMs": 831880,
    "text": "and taking us through that."
  },
  {
    "startMs": 833400,
    "text": "So,"
  },
  {
    "startMs": 834680,
    "text": "now that you've seen it work, let's look"
  },
  {
    "startMs": 836400,
    "text": "at some quick facts."
  },
  {
    "startMs": 838520,
    "text": "Routing decision in under 200"
  },
  {
    "startMs": 840600,
    "text": "milliseconds per request."
  },
  {
    "startMs": 843320,
    "text": "It runs on a custom mixture of experts"
  },
  {
    "startMs": 845600,
    "text": "model purpose-built for routing."
  },
  {
    "startMs": 848240,
    "text": "Zero application code changes needed"
  },
  {
    "startMs": 850080,
    "text": "from you to get it to adopt."
  },
  {
    "startMs": 852280,
    "text": "And it's free and included, so you do"
  },
  {
    "startMs": 853960,
    "text": "not have to roll out your own router."
  },
  {
    "startMs": 856720,
    "text": "And we open source the whole routing"
  },
  {
    "startMs": 858240,
    "text": "model via plain old, so you can actually"
  },
  {
    "startMs": 860440,
    "text": "check how that looks as well."
  },
  {
    "startMs": 864400,
    "text": "The last thing I wanted to talk about"
  },
  {
    "startMs": 866040,
    "text": "was a bit about"
  },
  {
    "startMs": 867560,
    "text": "routing is the foundation layer. It's"
  },
  {
    "startMs": 869440,
    "text": "not really the destination."
  },
  {
    "startMs": 871480,
    "text": "And there are three things that we"
  },
  {
    "startMs": 872720,
    "text": "usually build on top of it."
  },
  {
    "startMs": 874920,
    "text": "The first one is evals to prove that the"
  },
  {
    "startMs": 877360,
    "text": "right model works with your use case and"
  },
  {
    "startMs": 879520,
    "text": "your test well."
  },
  {
    "startMs": 881520,
    "text": "Caching, so that you can stop paying"
  },
  {
    "startMs": 883640,
    "text": "twice or more for the same answer each"
  },
  {
    "startMs": 886360,
    "text": "time."
  },
  {
    "startMs": 887680,
    "text": "And personalization, so that the router"
  },
  {
    "startMs": 889960,
    "text": "learns what works for your team over"
  },
  {
    "startMs": 892240,
    "text": "time."
  },
  {
    "startMs": 893280,
    "text": "This is a continuous improvement loop"
  },
  {
    "startMs": 895400,
    "text": "maturing over time. That means that the"
  },
  {
    "startMs": 897680,
    "text": "more you route and evaluate, the better"
  },
  {
    "startMs": 900000,
    "text": "the router does for your workload."
  },
  {
    "startMs": 904120,
    "text": "So, to summarize, where does this leave"
  },
  {
    "startMs": 906240,
    "text": "you?"
  },
  {
    "startMs": 907240,
    "text": "There is no single best model. There's"
  },
  {
    "startMs": 909360,
    "text": "only the right model for the request."
  },
  {
    "startMs": 912320,
    "text": "And benchmarks will only tell you part"
  },
  {
    "startMs": 914240,
    "text": "of the story. Your preferences will tell"
  },
  {
    "startMs": 916240,
    "text": "you the rest."
  },
  {
    "startMs": 917640,
    "text": "And we built the router to honor your"
  },
  {
    "startMs": 919400,
    "text": "preferences and stay open, so that"
  },
  {
    "startMs": 921760,
    "text": "you're never locked into a single stack."
  },
  {
    "startMs": 924600,
    "text": "And that's how teams actually built."
  },
  {
    "startMs": 927000,
    "text": "We are DigitalOcean, any i-native cloud."
  },
  {
    "startMs": 930120,
    "text": "Come find us at the booth and route your"
  },
  {
    "startMs": 932040,
    "text": "next workload with us. Thank you so much"
  },
  {
    "startMs": 933960,
    "text": "for being here."
  },
  {
    "startMs": 936005,
    "text": ">> [applause]"
  }
]
