AI Engineer Europe 2026
Build & deploy AI-powered apps
Loading slides…
Read the talk
Build & deploy AI-powered apps
From timestamped dinosaur videos to a persistent bookshelf catalog, Paige Bailey demonstrates how models, grounding, executable tools and app infrastructure work together.
From a talk by Paige Bailey
Before you start: Basic familiarity with model APIs, prompts and web applications is helpful; the examples use Python and TypeScript.
Which model belongs in the application?
How do you choose a model when an application needs to understand video, retrieve facts, generate media and save user data? The starting point is the work each component must do. Paige Bailey introduces the workshop as a Google DeepMind developer-relations engineering lead whose machine-learning work began in 2009, including contributions to NumPy, SciPy and scikit-learn. Her move through product and back to engineering reflects the increasingly overlapping work of designing, building and explaining these applications.
The release lineup separates several jobs. Gemini 3.1 Flash Live handles real-time interaction; Gemini 3.1 Pro and Gemini 3.1 Flash Lite offer different capability, cost and latency profiles. Bailey cites Augment Code as an adopter of Pro for performance and cost, describing a wholesale default-model change. Augment’s published account, subsequently updated, describes adding Gemini alongside other providers rather than replacing the entire system.
- Images and retrieval: Nano Banana 2 generates and edits images. A shared multimodal embedding space connects video, images, audio, text and code: a search for cats could retrieve a cat video, purring audio and books about cats.
- Generated media and worlds: Lyria 3 produces music; Genie 3 generates interactive worlds.
- Applications and deployment choices: AI Studio’s full-stack runtime adds databases and OAuth. Gemma 4 supplies an open-model family, while Veo 3.1 Lite supplies video generation.
Input and output modalities shape the application architecture. Gemini can ingest video, images, audio, text and code, including documents such as PDFs with embedded images. Depending on the selected model, outputs can include text, code, audio, images or interleaved images and text. Bailey contrasts that breadth with systems limited to text and code outputs or static-image inputs; the practical choice is to select the model and interface that support the particular combination the app needs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn a dinosaur video into a cited table
Google AI Studio provides the interactive starting point. Bailey signs in with a personal Gmail account, selects a model through the modality filters and chooses Flash Lite for speed. The tool controls expose structured outputs, code execution, function calling, Google Search grounding, Google Maps grounding and URL context. URL context brings supplied web pages into the model’s available context; search and retrieval provide information that may not exist in its training data.
The first concrete task is to identify the creatures in a dinosaur video and attach useful facts. AI Studio accepts media from Drive, uploads, recorded audio, a camera, sample assets or a YouTube link. With YouTube video input, Bailey can supply the URL directly rather than downloading and uploading the recording. She enables Google Search grounding before adding the video.
The selected interval is 0–300 seconds, sampled at the interface’s default of one frame per second. AI Studio reports approximately 27,600 tokens for this five-minute video input. That is the displayed total for this example; token accounting depends on the media configuration. The request is to create a table of the dinosaur types, their timestamps and a fun fact about each.
The dinosaur choice follows Bailey’s Easter outing, riding a tricycle down a winding San Francisco hill in a dinosaur costume. In the model request, the video supplies the visual evidence while Google Search supplies supporting information for the fun facts. The result identifies Rexy and his parents, Brachiosaurus, Velociraptor and Pteranodon. It also preserves a useful correction: Pteranodons are pterosaurs, not dinosaurs. Citations accompany the externally sourced facts.
Get Code is the bridge from experiment to application. It exports the selected model, the YouTube URI handling and the prompt in languages including Python, TypeScript and Java. The same workflow can accept an owned video instead of a YouTube URL. Bailey then shows the thinking controls—minimal, low, medium and high—and usually favors minimal or low for interactive latency. Higher thinking allocates more tokens to reasoning; her recommendation is to try Flash Lite’s speed and cost profile before reaching for a larger model.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give smaller models executable vision tools
Code execution gives Gemini a hosted Python sandbox with preinstalled data-science libraries. Generated code runs there rather than in the developer’s local environment. Bailey enables it on both sides of compare mode: Gemini 3.1 Flash Lite Preview on one side and Gemini 3 Flash Preview on the other.
Both models receive the same Lego photograph and the same task: draw bounding boxes around every green brick using Python, then display the annotated image. Bailey describes the image input as approximately 1,000 tokens. Flash Lite produces its annotated result first, generating Python to load the image, analyze it and define the boxes. The displayed result contains boxes around two green bricks.
A compact Python implementation of this kind of color-localization task separates the color mask, connected regions and final drawing. Here, the hue thresholds select green pixels; the area filter removes small regions before drawing each remaining region’s bounding rectangle:
python
import cv2
image = cv2.imread("lego.png")
if image is None:
raise FileNotFoundError("lego.png")
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, (35, 60, 40), (85, 255, 255))
contours, _ = cv2.findContours(
mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)
annotated = image.copy()
for contour in contours:
if cv2.contourArea(contour) < 100:
continue
x, y, width, height = cv2.boundingRect(contour)
cv2.rectangle(
annotated, (x, y), (x + width, y + height), (0, 0, 255), 2
)
cv2.imwrite("green-bricks.png", annotated)
These thresholds are an implementation choice for a color-based approach; touching bricks can merge into one region. The larger mechanism is that the model can turn its interpretation into an executable operation and return an inspectable artifact.
Bailey reports that the Flash Lite Lego run cost well under a fraction of a penny. She also suggests segmentation masks and counting selected entities as related uses of the same tool. Gemini 3 Flash eventually reaches the same answer; Bailey reports that its run took longer and cost slightly more, while remaining under a penny. These observations concern the two demonstrated runs, with code execution enabled on both models.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Supply recent sources, then choose a model tier
For a more focused retrieval task, Bailey turns on URL context and supplies blog posts about Gemma 4 and Genie 3, describing both as outside the model’s training cutoff. She asks it to compare similarities, differences and aspects that are unrelated, using medium thinking. The response lists the supplied sources and cites them inline as it makes assessments. The important distinction is between asking the model to remember a release and giving it the release information to read.
URL context is useful when the relevant public pages are already known. Bailey also points to Vertex retrieval tools for internal documents without necessarily provisioning a vector database yourself. Get Code exports the URL-context experiment just as it exports the video experiment.
Her positioning of the Gemini tiers is practical rather than a scored benchmark:
| Model | Position in Bailey’s workflow |
|---|---|
| Gemini 3.1 Pro Preview | Largest, more expensive and slower |
| Gemini 3 Flash | Production workhorse |
| Gemini 3.1 Flash Lite | Smaller option gaining users from earlier Flash models |
She specifically mentions users moving from Gemini 2.0 Flash and Gemini 2.5 Flash toward Flash Lite. The earlier Lego comparison shows why tools belong in that evaluation: model size alone does not determine how effectively a particular task gets completed.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep screen, speech and camera in one conversation
Gemini Live accepts screen sharing, video and audio in a dynamic conversation, with support for custom function calls, automatic function responses and Google Search grounding. Bailey enables Search, shares her screen and asks what the model sees. It describes the Google image search for Lego bricks, including the larger image of red, yellow, blue and green bricks on the right.
Next she recruits an Italian-speaking attendee, asks Gemini to repeat the description in Italian and adds a request for London’s weather that day. It responds in Italian with the Lego description and a cloudy-weather forecast with possible rain. The attendee approves the Italian. Bailey then asks for a poem about the same screen in a Texan accent, and the model changes its delivery while retaining the visual subject. Language, task and speaking style can change within the conversation.
A practical version is a bank-entry kiosk that responds in the language a visitor starts speaking. Where the product requires a fixed language, dialect, accent, style or tone, Bailey recommends setting that behavior in system instructions. Conversational requests provide flexibility; system instructions establish the intended application behavior. Get Code exposes the Live configuration, including media-resolution and compression settings.
She then switches to camera input. Gemini identifies two raised fingers and, after the gesture changes, a thumbs-up. Bailey contrasts this integrated interaction with separately assembling speech-to-text, an LLM, text-to-speech and video handling, and describes Live as a lower-cost option. The architectural benefit is that those inputs and responses participate in one continuing interaction rather than requiring the developer to coordinate every stage independently.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Ask for a bookshelf catalog with authentication and storage
AI Studio Build moves from model experiments to applications. Bailey compares the experience with v0 and Lovable: create, edit, share and deploy an app, with support for databases, Google login and securely held custom API keys. The gallery includes Guillaume’s Lyria Studio, Nano Banana examples and a MediaPipe game controlled by camera-based hand tracking. Each app’s source can be inspected.
For a fresh application, she selects Firestore and Firebase Google authentication, then dictates the requirements using the microphone:
- Let the user upload a bookshelf photograph with visible book spines.
- Identify titles and authors, using Google Search grounding to add descriptions and categories.
- Ask the user to sign in with Google.
- Save the catalog for that user in a database so it persists.
The requested application combines visual recognition, external enrichment, identity and durable storage.
The requested data can be expressed as a small TypeScript contract. Keeping the owner separate from the book metadata makes the per-user requirement explicit:
typescript
type Book = {
title: string;
author: string;
description: string;
category: string;
};
type Catalog = {
ownerId: string;
books: Book[];
};
function createCatalog(ownerId: string, books: Book[]): Catalog {
if (!ownerId.trim()) {
throw new Error("A signed-in user is required");
}
return {
ownerId,
books: books.map((book) => ({ ...book })),
};
}
This represents the data passed toward persistence; enforcing ownership also requires the database access rules. At this point in the demonstration, the catalog and its persistence are requested behavior, not a completed application.
Bailey observes that the request could have constituted a startup a few years earlier, then clicks Build. Gemini 3.1 Pro Preview begins planning the app, database and authentication setup. The work continues in the background while she demonstrates other models; an audible notification signals when it needs her attention.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Navigate a world generated as pixels
While the app builds, Bailey opens Project Genie. Its Genie 3 world model generates frames dynamically as the user explores a described environment with a described character. Movement uses WASD, viewpoint changes use the arrow keys, and the space bar provides additional actions. Bailey emphasizes that this experience does not have a conventional Unity, Unreal or physics engine underneath it.
The existing examples include a rover in a volcanic landscape, a jet ski, a Florida hurricane and an underwater jellyfish scene. In the jet-ski example, striking an object produces an apparent collision response. Bailey attributes that behavior to learned world behavior rather than a separately programmed physics simulation. A notification interrupts the tour: AI Studio needs Firebase database enablement for the bookshelf app. She handles that checkpoint, then signs into Project Genie to create a world.
Her environment prompt describes Big Bend National Park in summer, with sunshine, sandy ground, Lego-brick rock formations and a quadruple rainbow. The character is a pink ostrich with goggles and a rocket blaster. Bailey describes a composition of models involving Nano Banana, Veo and Gemini prompting. More precisely, Project Genie is the application around Genie 3; Google’s launch description names Genie 3, Nano Banana Pro and Gemini as components, without naming Veo.
The preview produces Lego formations and an ostrich that has also acquired muscular arms. Bailey creates the world, walks the character among the rocks, changes the view and heads toward a canyon where she hopes to find a river. The attempted canyon jump is not completed before the exploration ends. She also mentions an unnamed colleague’s example of a fish trying to escape a kitchen. At the time of the workshop, she says Genie 3 has no public API, a trusted-tester program is under consideration, and access through an Ultra subscription is limited to selected countries.
An audience question identifies the key boundary: can this world be imported into Unity? The generated experience supplies pixels, not exportable 3D meshes. There is no ostrich asset to extract directly. Turning a generated image into a 3D asset requires another process. Bailey contrasts this with World Labs’ approach to world building and suggests that video input, 3D worlds and generated code may eventually converge. That is a possible future direction, not a capability demonstrated here.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Ask for a vector representation of the Lego image
The discussion of generated assets prompts an improvised image-to-code test. Bailey reuses the Lego photograph and asks Gemini to create an SVG representation of it. The initial prompt is deliberately simple; if the result is weak, she proposes asking Gemini to expand the prompt.
She recalls a Nano Banana hackathon project that turned an input image into step-by-step drawing instructions with successive stroke marks. In the current SVG test, Gemini 3.1 Pro runs with high thinking and displays planning about perspective, brick dimensions, a grid, colors, rotations and transformations. While it works, the bookshelf app requests attention again: Bailey accepts the Firebase terms and selects Retry. TypeScript and CSS have been generated, but the app is not finished yet.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Combine reference images with a searched object
Nano Banana 2 adds another kind of grounding: image search. Bailey enables the image-search option, which she describes as reverse image search, and supplies two sample images—a dog and a natural park. The request is to place the dog in the park with a can of Celsius. The supplied images establish the subject and setting; search is intended to supply details of the branded can.
After joking about Celsius as a popular but, to her taste, unpleasant hackathon drink, Bailey inspects the generated dog-in-park scene with the can. She describes Nano Banana 2 as more cost-effective than the Pro tier and earlier iterations, without giving an exact price. Get Code again exports the configuration for TypeScript or Python. An audience suggestion about speed returns to the thinking control: minimal or low thinking gives a quicker response, while more reasoning spends additional tokens planning the task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Inspect the SVG, then add code execution
Returning to the SVG, Bailey copies the generated code into an online visualizer. The result contains recognizable Lego bricks, but they are distorted and arranged differently from the photograph. A more detailed prompt might improve the reconstruction, but that improvement has not yet been demonstrated. She instead enables code execution to see whether executable tools can help with the geometry.
The model starts using a library and performs segmentation. The displayed intermediate result isolates a blue six-stud brick on coordinate axes, with red points around its outline. This gives the reconstruction process an inspectable geometric intermediate; the demonstration does not establish a finished, improved SVG. The tool remains the hosted Python sandbox, accessible through the API, with preinstalled data-science libraries.
The bookshelf app is still progressing in parallel. Bailey checks its code view and finds the Firebase blueprint and rules being generated. Those backend artifacts are part of the requested application, alongside its visible interface.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate footage from an expanded creative brief
For Veo 3.1 Lite, Bailey first asks Gemini to write the video-generation prompt. With thinking set to low, she requests stock footage for a vegan basketball-themed food truck whose menu references the Golden State Warriors. She then copies Gemini’s expanded prompt into Veo. This makes prompt development a separate step from generating the final video with audio.
The demo uses 720p output. Bailey describes landscape and portrait aspect ratios of 16:9 and 9:16, with four-, six- or eight-second durations; she says 4K requires a higher-tier video model. Veo is a paid-tier model in this workflow and needs an appropriate API key. AI Studio’s Get API key section can also create a key for eligible free-tier uses, but that does not make the Veo request free.
The generated footage includes Chef Curry and Splash Brothers references. Bailey identifies tofu barbacoa, kale, avocado and edamame in the food imagery. The result connects the visual style, menu and basketball brief rather than producing a generic truck scene. It also has a personal destination: someone she knows wants to start a vegan basketball food truck, alongside a possible nut-butter business.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluate an open model before running it locally
Gemma 4 offers a different deployment choice. Bailey presents AI Studio and its API as a way to evaluate the models for free before downloading them, including for developers without a spare GPU. She supplies an example image and requests a brief description, with thinking set to minimal.
The Apache 2.0 license permits downloading, fine-tuning and running the models on owned infrastructure. Modality support needs to be checked by variant: the launch documentation assigns image and video understanding across the four launch models, but native audio specifically to E2B and E4B. That distinction refines Bailey’s family-level description of audio, video and image understanding.
The smallest versions also target mobile execution. Bailey discusses local Pixel use and anticipates Gemma inclusion in Pixel 10; that is an expectation expressed in the workshop. Her subsequent Chrome reference needs a naming distinction: Chrome’s documented built-in language model is Gemini Nano, rather than Gemma.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give the food truck a Spanish theme song
With the bookshelf app still building, Bailey opens the audio options: Lyria 3 Pro Preview and Lyria 3 Clip Preview. Templates include acoustic folk and ’90s alternative rock, but she uses Guillaume’s gallery app to configure the sound. Its source is available to inspect and remix.
The brief carries forward the vegan basketball food truck and adds Lego bricks. Bailey selects an electronic, danceable clip, then asks an attendee for a language and requests Spanish lyrics. The generated playback refers to the vegan food truck, a basketball team, colorful Lego pieces and a world of flavors, with the refrain “La fiesta vegana.” The same creative concept now has both video footage and a theme song.
The gallery app provides a concrete route for experimenting with Lyria’s controls and implementation. Bailey points to the earlier generative-media session and a later workshop for more depth, while emphasizing that the app’s code is available for further experiments.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Test authentication and persistence before sharing
The bookshelf cataloger is finally ready to exercise. Bailey signs in with her personal Google account, then selects a small bookshelf photograph. It includes handwritten-style title text and partly hidden author names, making it a useful test of the requested recognition and enrichment behavior. She uploads the photograph; the application displays book names, authors and descriptions.
She then signs out and signs back in. The books remain in the catalog. The demonstrated persistence check is a session round trip: the user returns through authentication and sees the previously created records. The selected frame shows the populated catalog beside the sharing panel, which is still set to restricted access at that moment.
Bailey subsequently changes the app to public, copies its link and displays a QR code for the audience. Public sharing and Cloud Run deployment are separate actions. She describes one-button Cloud Run deployment but explicitly declines to perform it for this app to conserve her quota. The completed demonstration establishes sign-in, catalog creation, persistence across sign-out and sign-in, and public sharing—not a Cloud Run deployment.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate planning from action in the systems around you
The closing robotics example makes an architectural boundary explicit. Bailey presents Pupper as an open-source Raspberry Pi robot with printable parts, using Gemini for object detection and responses to its environment. The Stanford Pupper control repository provides background on that robot family. In the architecture she describes, Gemini Live interprets flexible instructions and builds a plan, then invokes local robot models to perform actions such as picking up an item. It does not directly control the robot’s movements.
The same multimodal interaction can support augmented-reality assistance: directions, descriptions of visible surroundings, explanations of math on a whiteboard, or English transcription of someone speaking Chinese. The model’s role is to interpret the incoming context and provide the appropriate instruction or representation.
In the final Q&A, an attendee asks about a native Google AI application comparable to Codex. Bailey says an AI Studio app is planned and refers to Logan’s public hints, without giving a release date or promising a Codex equivalent. For existing workflows, she points to using Gemini APIs with OpenClaw and describes a colleague’s Telegram-plus-Gemini setup for workspace actions and Google Search. Free tiers for Gemma and some Gemini models provide an entry point for those integrations: the same model capabilities explored in AI Studio can become components in the tools people already use.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
- Gemma 4 launch announcementArticle
Introduces Gemma 4’s launch models, Apache 2.0 license, modality support and local deployment options.
- How Project Genie worksArticle
Explains world sketching, interactive exploration and remixing, with launch access requirements and prototype limitations.
Explains Gemini Nano in Chrome, supported APIs and local hardware requirements.
- Stanford Pupper control softwareRepository
Original Raspberry Pi quadruped control software and build-guide links, with an end-of-life notice for Pupper v1.
Updates since the talk
Current guidance for video inputs, YouTube URLs, frame sampling and token accounting.
Instructions for generating apps with server-side secrets, Firebase authentication, persistent storage and Cloud Run deployment.
Documentation for model-generated code execution, available libraries, billing and limitations.
Full transcript
Loading transcript…