AI Engineer Europe 2026

GPU Cloud Deployment Without Leaving Your IDE — Audry Hsu, RunPod

Audry Hsu20:19

Loading slides…

Read the talk

GPU Cloud Deployment Without Leaving Your IDE

Audry Hsu demonstrates how RunPod Flash moves Python inference functions onto cloud GPUs, then extends image generation into a pipeline with prompt expansion and photo composition.

From a talk by Audry Hsu

Before you start: Familiarity with async Python functions, HTTP requests and the distinction between local code and remote GPU execution will help you follow the demonstration.

From model training to infrastructure work

Yunus, an Oxford attendee, already has a practical reason to use RunPod: his university provides credits, and he spends them on LLM training. That is the starting requirement—access to compute so he can work on a model.

RunPod supplies the GPUs and compute; developers bring their code and models. Audry Hsu describes the work that gets between those two things: aligning CUDA and PyTorch versions, testing new GPU SKUs, resolving hardware-related bugs, and configuring deployment and scaling. Infrastructure configuration competes directly with model development. The platform’s purpose is to move that work out of the developer’s path.

Slide titled “Why Runpod Exists” with three cards about developer time, slow and opaque GPU access, and focusing on building.Open full source frame
Why Runpod exists: infrastructure work and slow GPU access distract developers from building.
0:350:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:35 · section reference included

From spare GPUs to deployment options

Hsu traces RunPod’s origins to Zhen and Pardeep, who founded the company in 2022 with basement GPUs left over from a failed crypto-mining venture. They built a prototype and offered free GPU access on Reddit in exchange for feedback. That became a pattern of building in public with the community; Hsu says the company generated revenue from the beginning.

Hsu reports around 500 developers on the platform, more than 30 data centers across 10 countries, and $120 million in annual recurring revenue. She names France, Romania and Iceland within the European footprint, and also mentions Asia Pacific. RunPod’s company announcement separately supports the ARR milestone as a company-reported figure. The customers shown span AI-native companies and large enterprises, connected by a need for flexible, reliable GPU infrastructure.

The product choice depends on how the workload uses that infrastructure:

ProductWorkload fitOperating model
PodsPersistent development or compute environmentRent on demand, pay by the second, tear down when finished
ServerlessVariable request frequency and loadAutoscale workers up and down
ClustersMulti-node trainingCoordinate compute across nodes
HubQuick explorationDeploy pre-vetted open-source repositories

A running Pod keeps its GPU reserved for that Pod. Serverless instead adjusts worker capacity to demand; Hsu introduces scale-down as the way to avoid leaving unused capacity running. Hub offers starting points such as ComfyUI, Stable Diffusion and vLLM. The demonstration follows the Serverless route.

2:172:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:17 · section reference included

Keep the development loop local

During inference development, testing a small code change can require a much larger deployment procedure:

  1. Commit the change and push it to GitHub.
  2. Build a Docker image.
  3. Pull the image from a container registry onto a server.
  4. Allocate a GPU.
  5. Test the result, then repeat.

RunPod Flash, a Python SDK, is designed to remove those manual steps from the iteration loop so a function can run on a cloud GPU directly from the local development environment.

The execution boundary is an ordinary async Python function with a Flash endpoint decorator. Flash packages and deploys that function to the GPU cloud, while the surrounding main function and helper functions remain local. File reload repackages and pushes changes so the developer can test again without manually rebuilding and uploading a container. The GPU function moves; local orchestration stays local.

The recording uses the then-demonstrated endpoint decorator and flash run. The current repository documents @Endpoint and commands including flash dev and flash deploy; those should not be read as the exact API shown onstage.

Runpod Flash documentation with a Python code block, explanatory paragraph, and Quickstart, Create endpoints, and Examples cards.Open full source frame
Flash documentation shows a Python example and describes remote execution alongside local code.
5:435:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:43 · section reference included

Send an image request through the local server

The first GPU function generates an image with PyTorch and pretrained Stable Diffusion XL Turbo, selected for fast image generation. It saves the image and returns a Base64-encoded representation. The project and dependencies are already installed when Hsu begins running it.

Hsu invokes flash run, which starts a local FastAPI development server. She refers first to image_generation.py and later to image_generation_async while finding the script to run. A local helper sends a POST request to the development endpoint, then decodes the returned image so it can be displayed. The serialization boundary can be expressed with two small Python helpers:

python

import base64
from pathlib import Path


def encode_image(path: Path) -> str:
    return base64.b64encode(path.read_bytes()).decode("ascii")


def save_image(encoded_image: str, destination: Path) -> None:
    destination.write_bytes(base64.b64decode(encoded_image))

The GPU side returns text representing the image bytes; the local side reconstructs a file from those bytes.

The audience supplies the subject: cats flying through a cloudy London sky. Before that prompt reaches the model, the request needs live troubleshooting around its URL, localhost and HTTP. Once the request gets through, the local server reports that the job has started and entered the queue. This separates two questions that are easy to conflate during a demo: whether the request reached the service, and whether it carried the intended model input.

7:427:59
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:42 · section reference included

Configure workers, correct the prompt, swap the model

While the job runs, Hsu returns to the endpoint decorator. It carries the endpoint name, GPU-family selection and worker settings. Her spoken GPU-pool name corresponds closely to ADA_80_PRO, which the GPU pool reference maps to H100 GPUs with 80 GB of memory; the pool identifier is not an architecture name. The demonstration sets a maximum of five workers and one active worker that stays running.

The first output is a dragon. The service has run, but the intended cats prompt was not passed as a flag. Hsu adds the prompt flag and submits again, then points out the idle-timeout setting, which controls how long a worker remains idle. The corrected request produces cats, but their abstract appearance is not the result she wanted.

The next edit changes the model rather than the request path. Hsu comments out the original implementation and substitutes DreamShaper, which she describes as a Stable Diffusion 1.5 fine-tune. Her working comparison is XL Turbo for quick generation versus DreamShaper for the artistic and illustrative quality she wants here. She sets DreamShaper to 25 inference steps, reviews the size parameters, and resends the same cats request.

The change stays inside the IDE: edit the function, let Flash propagate it, and send the request again. There is no manual commit, Docker rebuild, upload or GPU-allocation sequence between those two model trials. Hsu and the audience prefer the new image. The useful result is a shorter development procedure and a preferred output in this session; no timed comparison or controlled image-quality evaluation accompanies it.

11:1411:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:14 · section reference included

Add prompt generation and photo composition

A single remote model call is only part of an application. The next example uses prepared orchestration code to connect three stages:

  1. Call Qwen 3 on an already-hosted public endpoint to generate richer image prompts.
  2. Send those prompts to DreamShaper on the Flash-managed endpoint.
  3. Pass the images through Nano Banana 2, Google’s Gemini 3.1 Flash Image, for photo composition.

Hsu’s intended subjects are RunPod’s founders. This extends the local orchestration pattern from calling one GPU function to coordinating prompt generation, image generation and an external composition model.

The audience shapes the new input into two men with glasses walking in London on a cloudy day, with a close-up of their faces. Hsu requests three generated images and starts the pipeline, leaving it running while she answers a pricing question.

14:3414:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:34 · section reference included

Understand what stays running

Asked about pricing, Hsu initially describes a charge for the time each request runs. She then leaves the terminal to inspect the endpoint in the web console. She describes roughly five or six provisioned workers and three running for the three requested photos, before pointing to worker uptime as the billed quantity and discussing an H100 per-second rate.

Runpod console for “image generator - SDXL turbo,” showing a worker table with location, GPU type, uptime, and version columns.Open full source frame
The endpoint’s Workers tab lists GPU workers and their uptime.

Worker lifetime, not just inference duration, is the useful cost boundary. Current Serverless billing documentation specifies charges from worker startup through full stop, including initialization, execution and idle timeout; active workers run continuously. That qualifies the request-only shorthand and matters for the always-on worker in this example. Hsu explains Serverless’s premium over Pods through the scaling it provides. Her recommendation is to start experimentation with a low worker count or a Pod when only one or two GPUs are needed, then use Serverless when the application needs hundreds of workers and distribution across data centers for availability.

16:5517:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:55 · section reference included

Inspect the generated and composed images

The finished presentation separates the intermediate image from the final composition. On the left is DreamShaper’s output from Qwen 3’s expanded version of the original London prompt. The expansion adds thoughtful expressions, weathered faces, soft-focus background clouds, a muted urban palette of grays and deep blues, and overcast lighting. Qwen contributes visual direction before DreamShaper turns it into an image.

On the right is the composed photo of Pardeep and Zhen. Hsu jokes that Pardeep looks handsome while Zhen looks unexpectedly old, then scrolls to the reference photo supplied to the composition stage. Showing that reference makes the final operation easier to understand: the pipeline combines the generated scene with a separate photographic input.

The final visible result pair shows generated and composed images beneath golden trees in Hyde Park, London. The workflow still begins in the local development environment, even though the application coordinates remote inference and composition. Hsu closes by extending that starting point beyond the open-source models shown: developers can also bring their own private models.

Two image cards labeled generated and composed show people beneath golden trees in Hyde Park, London.Open full source frame
Generated and composed Hyde Park images appear side by side.
18:4018:49
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:40 · section reference included

Resources

From the talk

  • Lists GPU identifiers, memory capacities and pool mappings, including ADA_80_PRO for H100 GPUs.

Updates since the talk

Transcript download
Full transcript

Loading transcript…