[
  {
    "startMs": 12280,
    "text": "Hello, good afternoon, everyone. I'm, uh, Dean Quiñanola from Runpod Flash, and, um, I wanna talk to you about how you might be able to deploy AI applications without Docker. So currently, this is how you deploy a serverless application. Well, especially AI applications. You write code, you build your Dockerfile, you run a bunch of commands, Docker build, all that, and then you"
  },
  {
    "startMs": 42220,
    "text": "push the image to Docker Hub. Then you execute it. Then maybe you'll get what you want. If, if not, you have to change your code again, rebuild it, re-push it, execute. Um, in my experience, sometimes this one loop takes about fifteen minutes depending on the changes you're doing. So my name is Dean Quiñanola. I am a staff engineer at Runpod, and I have an entire team that's working on making that easy."
  },
  {
    "startMs": 73040,
    "text": "So why did we build Flash? Development iteration loop requires a makeover. It's really a hassle. A lot of you guys are software engineers or ML engineers who really don't wanna deal with all the DevOps. So as you're working on your ideas or you... as you're working on your experiments, you want the results. So no more need for Docker. Run code like it's local, as if you have the GPU. Uh, test your changes instantly, and debug"
  },
  {
    "startMs": 102280,
    "text": "quickly if you need to. So this is how we're thinking about keeping it simple. You write your code, you execute. You get the results. Rewrite your code, execute, you get the results, right? Of course, there's the deployment part. I'll talk about that later."
  },
  {
    "startMs": 122640,
    "text": "So it's so simple, it feels like it's running locally. So what are the features that we have? So we are able to build and deploy faster. We optimize the resource utilization. There's a lot of tricks happening under the hood, and it's intuitive. It's just Python code. It's really whatever you can run locally with a GPU, just do it on the cloud. So what can you do with Flash today? We have serverless endpoints. It's mostly locked on serverless right now, but we're working towards doing it for pods."
  },
  {
    "startMs": 152960,
    "text": "Uh, we have queue endpoints for it and load balance endpoints that you can use it for. Here's the, uh, really cool feature that I put in, an endpoint-to-endpoint communication. Sometimes you're building all these servers, but then you still have to know how to contact each server, right? You don't have to do that with this 'cause it's just a code talking to another code. You execute a function, and it looks like it's just local. But once you deploy it in different endpoints, it's all talking to each other. It's all"
  },
  {
    "startMs": 182840,
    "text": "happening under the hood. We also have network volume attachments. Um, we have warm caching in network volume attachments, so that if you ever have to scale up with your serverless endpoints, then it has those cache carried over through the network volumes. And then the most important part is the live serverless. Uh, you are coding and testing as if it's all happening in your local system. And then, you know, it-- this would just be a toy project if you can't deploy to production, right?"
  },
  {
    "startMs": 213020,
    "text": "So you wanna deploy to production, so serve the inference, serve all the, uh, stuff that you're building, and we're making that possible all through the CLI. So let's get a demo. Hopefully, I can do this with one hand."
  },
  {
    "startMs": 228680,
    "text": "Okay. All right. So my demo here shows the, the rapid iteration and evolution of a function. So the first function is just talking to Torch, getting what kind of GPU you have. So I'll do this live. I'll do flash"
  },
  {
    "startMs": 249200,
    "text": "dev. And what this does is it launches a FastAPI localhost, so you can actually toy around with it. So now you wanna try it out. Let's try it out. Execute. So it's provisioning this serverless endpoint, and if we go to Runpod console, refresh this, you'll see this. This is running now."
  },
  {
    "startMs": 273240,
    "text": "And as it's loading,"
  },
  {
    "startMs": 276220,
    "text": "let's, let's show the logs so you can actually see it."
  },
  {
    "startMs": 284980,
    "text": "All right. Worker is ready."
  },
  {
    "startMs": 292680,
    "text": "Yep."
  },
  {
    "startMs": 302760,
    "text": "Oh, print job."
  },
  {
    "startMs": 306800,
    "text": "Cold start. This is common with all serverless platforms."
  },
  {
    "startMs": 313280,
    "text": "Oh, there it is. So it's running. We got logs, container logs. So it's testing all these things, making sure it's all healthy, and it ran the job. So now if we go back here, you'll see the result. It ran on an NVIDIA G-Geforce RTX forty ninety. You see all the RAM, GPU count. Now, here's the kicker. What if I wanna change this function and just wanna test it? So here's a different"
  },
  {
    "startMs": 343040,
    "text": "iteration of this function. What we do is we add more code, and this code really just does generation of text. And I've added some dependencies, the Python dependencies there. Oh, actually, let's do this without the Hugging Face token first. Let's see what's happening. See, it refreshed everything. I refresh this. Test it again."
  },
  {
    "startMs": 371130,
    "text": "What prompt do we need? Uh, let's do string cheese. Is it real?"
  },
  {
    "startMs": 379670,
    "text": "Run it."
  },
  {
    "startMs": 382590,
    "text": "You see it again here."
  },
  {
    "startMs": 390230,
    "text": "Container. Okay,"
  },
  {
    "startMs": 393750,
    "text": "it's coming up. You'll see all the... The, the logs are simpler here because you're not supposed to be inundated with so much stuff. But what it's doing is it's, on the fly, installing the dependencies, and it even gave me a warning about communicating with the Hugging Face server without the token, and it should be finished. Okay, now it's finished, let's see the result. I ask, \"Is string cheese real?\" \"I'm sorry, but I cannot provide answers.\""
  },
  {
    "startMs": 425410,
    "text": "All right, that's, that's spicy. All right, let's, let's change that up. Let's enable my HF token. That's just changing the code, right? Now, we release this again. Let's change the, uh, prompt."
  },
  {
    "startMs": 445670,
    "text": "Let's see."
  },
  {
    "startMs": 449590,
    "text": "Who is Batman? Let's execute that."
  },
  {
    "startMs": 455210,
    "text": "Ready. All right. Let's see. How is this working here?"
  },
  {
    "startMs": 461510,
    "text": "It's cool. I'm doing this with one hand."
  },
  {
    "startMs": 466330,
    "text": "All right, that's finished. Now we see the result. Oh, there you, there you go. \"Batman's a fictional character.\" All right. We knew that. Let's go for something that's more intensive than just generating text. Now we'll do"
  },
  {
    "startMs": 484390,
    "text": "an image generation. All right. It refreshed. Let's refresh this local host. Now you see generate image, and then we'll test this out with a prompt. Let's create an image of a string cheese Batman."
  },
  {
    "startMs": 509710,
    "text": "All right. This is how creative AI could be."
  },
  {
    "startMs": 515090,
    "text": "Let's look at what's happening here."
  },
  {
    "startMs": 520210,
    "text": "Running logs. Oh, look at that, rollout in progress."
  },
  {
    "startMs": 535530,
    "text": "Okay."
  },
  {
    "startMs": 539370,
    "text": "Oh, I got plenty of time."
  },
  {
    "startMs": 547130,
    "text": "That's not it. All right. I just had to terminate that 'cause it was running the, the older version. All right, now it's new. I didn't wanna wait for that rollout."
  },
  {
    "startMs": 560490,
    "text": "I just wanna see string cheese Batman. All right. So it, it responded with a base64. Okay, let me clean that up before I show that."
  },
  {
    "startMs": 574970,
    "text": "So"
  },
  {
    "startMs": 577450,
    "text": "how you can render this is through the browser."
  },
  {
    "startMs": 587670,
    "text": "All right. Let's see what string cheese Batman looks like."
  },
  {
    "startMs": 593590,
    "text": "That's not string cheese, but that proves it, right? It did it. Now, what if you wanna deploy this? So check this out. When I cancel it, it actually deprovisioned that endpoint, so you don't have to have that lying around. See, it's gone now. Now, what if we wanna deploy this? So let's deploy that."
  },
  {
    "startMs": 618170,
    "text": "Flash deploy. That packages up the whole thing, builds it up, sends it to our servers. Again, no Docker. You're not dealing with Docker here. This is instantaneous packaging of your application, sending it out to our servers, and serving it."
  },
  {
    "startMs": 638250,
    "text": "Now, here's the cool part. Once it's out there,"
  },
  {
    "startMs": 642770,
    "text": "I have prepared this prompt,"
  },
  {
    "startMs": 646870,
    "text": "but I just need to change this ID later 'cause this is gonna matter. All right, so it's created the archive, built Flash. All right, sending it out, uploaded, deploying. Now it's up. All right, let's see the ID here."
  },
  {
    "startMs": 667310,
    "text": "Okay. Now let's get this going. So my prompt is a cute baby sea otter with a bow tie."
  },
  {
    "startMs": 679550,
    "text": "Generate that image. Oh yeah, I wanna show you where this is. Now, this is fully servable, right? Like, you can have your customers, you know, use your application that uses this back end. It no longer lives in my local host. This is being hosted."
  },
  {
    "startMs": 697590,
    "text": "All right, let's see workers log."
  },
  {
    "startMs": 705010,
    "text": "Okay. When it's done, it should print the output here. There we go. All right. Let me wrap this so I could pull this up."
  },
  {
    "startMs": 722910,
    "text": "Whoo. This is crazy doing it in one hand. All right. Let's load up the picture of a sea otter with a bow tie. There's no bow tie"
  },
  {
    "startMs": 734150,
    "text": "But that's not the point. We... All right, so you get that. That's the demo. We had a hackathon yesterday, and I just wanted to be, to present this to you. I'm very proud of this guy, uh, Fong Cao. He created a wrapper around our Flash that enables you to upload a data sheet of all your experiments, of some specifications, of some models, and run it instantly. And it just fans out, builds out all these endpoints, runs"
  },
  {
    "startMs": 764110,
    "text": "your training, and then reports back to you. And he did this within three to four hours, and he had no idea how to do Runpod Flash. He had to ask me questions if this was even possible. I'm like, \"Let's see.\" And he was very surprised when he won yesterday  because his, his project was not even his best one, apparently, and he keeps losing to other hackathons, so just really proud of this guy. How do you get"
  },
  {
    "startMs": 794050,
    "text": "started with Runpod Flash? So you sign up for an account on Runpod, scan that QR code, get some $10, uh, credit now, and all you really have to do is just install it, write your code, log in to have your, uh, API key in there. Initialize, it'll have some skeleton application if you want. Flash dev, the local dev that you have, or Flash deploy to production. And we have all these URLs. The best advice I have is visit the Runpod"
  },
  {
    "startMs": 823810,
    "text": "Flash examples on GitHub because of we have all these cases and examples that you could kind of refer. Or if you like AI coding, just run Claude Code into that repository, and it will learn everything. We also have skills if you're into that. And that is my presentation."
  }
]
