[
  {
    "startMs": 12440,
    "text": "Can you hear me? Fantastic. Hello, everyone. Well, hello couple people that wandered in. Um, I'm sorry the schedule has been changing around. I'm sorry you didn't find the, a quiet working area you were looking  for. All right. Um, we are looking at some agents with Gemini API. Now, before we kick off, let's have a little listen to this."
  },
  {
    "startMs": 41220,
    "text": "Welcome one and all to the grandest stage on Earth. Ha. You're standing right at the heart of the largest AI engineer world's fair in history. Let's build the future together, shall we?"
  },
  {
    "startMs": 57000,
    "text": "Does anyone know what model that is?"
  },
  {
    "startMs": 61740,
    "text": "Yeah, you're, you, you, you're not allowed. Anyone else? Anyone know which model? No? Any guesses? It starts with G."
  },
  {
    "startMs": 77480,
    "text": "Gemini. Yes, it is a Gemini model, but which one?"
  },
  {
    "startMs": 84280,
    "text": "No, it's not Live. That's incorrect."
  },
  {
    "startMs": 88420,
    "text": "Yes, is it Gemini TTS? Is that what you were about to say? No, okay. Um, so Gemini Text to Speech, uh, actually a really incredible model for expressive speech. You have inline audio text you can steer it with. Um, you can stream the speech. It's, uh, a lot lower latency now than it used to be. So, uh, if you are in the market for some speech, uh, text to speech that is, Gemini TTS is a good"
  },
  {
    "startMs": 118240,
    "text": "model. Now, let's have a look at, uh, some of the things we're doing today. Uh, first of all, my name is Thor. I work on the developer experience at Google DeepMind, so that mostly includes Gemini API and Google AI Studio. So if you're using the Gemini models through the API, uh, you know, let me know if you have any feedback. Uh, I have my colleague Philip here as well, uh, who's working on managed agents on the API. So"
  },
  {
    "startMs": 148420,
    "text": "if you're using that, uh, let us know as well. So this is a bit of a, a, a pre-look at the workshop we're doing tomorrow. Um, so tomorrow we'll do kind of Gemini API, especially focused on, um, live agents, uh, conversational agents, real-time, uh, conversational agents, multimodal real-time conversational agents. We're just packing everything in there. Um, and so what we can look at, actually, there's,"
  },
  {
    "startMs": 178380,
    "text": "uh, been a lot of things that shipped. And so specifically what we want to look at is the multimodality of Gemini and the Gen Media models. Uh, so actually, this slide is already out of date because this morning we shipped a new model, uh, that is called... Anyone know which model did we ship? We shipped OmniFlash, which is, uh, a video generation and editing model. And we shipped NanoBanana Lite. So NanoBanana Lite is a really"
  },
  {
    "startMs": 208200,
    "text": "exciting model because it cuts down the latency of image generation and image editing quite significantly. And so what I've done, we'll, we'll do a couple demos here. Uh, and one of the demos is, um, this example here, which I call the, uh, Gemini NanoBanana story canvas. Uh, and I just vibe-coded this up, uh, just recently, and I'm having some trouble with my mouse here. Um, but so what we can do is we can"
  },
  {
    "startMs": 238420,
    "text": "come up with a story with, um, Gemini Live, and then have it use NanoBanana Lite to, to illustrate that story for us. So anyone, you know, this is a little bit like improv theater now. Anyone has a storyline they want to start with?"
  },
  {
    "startMs": 255880,
    "text": "Where are we? Somewhere... Okay, let's say, um,"
  },
  {
    "startMs": 263240,
    "text": "we're in Taipei in the 1980s."
  },
  {
    "startMs": 267180,
    "text": "The neon lights of Taipei's night market pulsed with an almost magical energy, reflecting off the wet pavement and the chrome of passing scooters. Rain had just passed, leaving a humid mist that hung in the air. Amidst the throng of shoppers and vendors, a young woman clutched a small, strangely glowing object under her jacket."
  },
  {
    "startMs": 288000,
    "text": "Something very exciting happened in 1980s Taipei. A new kind of drink emerged."
  },
  {
    "startMs": 294880,
    "text": "She paused at a stall where a man was vigorously shaking a metal cup, creating a frothy, milky concoction. He handed a cup to a customer, who took a long sip with wide eyes, savoring the strange tapioca pearls at the bottom. This was the birth of bubble tea, and its sweet, chewy magic was spreading like wildfire."
  },
  {
    "startMs": 318040,
    "text": "So what you can see here is, uh, from the tool call. So Gemini Live is getting a tool call.  Um, and then it's using the Gemini Interactions API to generate an image on the fly. Now, it still takes kind of, you know, two to three seconds for the actual image generation, but that is, um, pretty impressively fast. And, um, what you can look at is if you're looking at the code here, uh, we're using the Interactions API. And so the Interactions API"
  },
  {
    "startMs": 348100,
    "text": "is exciting because it is a stateful API, which means we can chain our interactions very seamlessly. So you could see here the first image generation that we were doing, uh, it didn't have a previous interaction ID- But then the next generation that we're doing, we're passing this previous interaction ID to the interactions API, which means that we have the previous context of the previous interaction, and we can continue that in the"
  },
  {
    "startMs": 377804,
    "text": "generation. So what that means specifically for our demo here is that you can see is our first, you know, interaction, uh, it generated an image with a very specific kind of visual feeling and also with characters, right? And so we can continue that interaction, and so we can keep the same consistency kinda across these generations. So if you haven't tried out the interactions API, uh, it is our unified API for interacting with all of the Google DeepMind models as well"
  },
  {
    "startMs": 407584,
    "text": "as agents. Uh, and so do try that out. It is a, it is a really, really cool, um, way to interact with everything that Google DeepMind has. Okay. So this was, you know, a bunch of things that happened. Um, so this was kind of on the Gen Media side. We shipped, uh, NanoBanana Lite this morning. Now, we also recently shipped on the audio side our live translation model. Has anyone tried out Gemini 3.5 Live Translate?"
  },
  {
    "startMs": 438184,
    "text": "Yes. Okay, a couple folks. Uh, that's fantastic. So what we can do is, uh, we can try that out as well. We can play, uh, a little demo. And so what I'm gonna do here is I'm gonna create this live translation session, and I want you to scan, uh, this QR code on your phone. And so what this is doing here is if I now enable, um, my microphone, so this is getting the, the"
  },
  {
    "startMs": 467604,
    "text": "microphone of my speech, um, here into the session. And now if you go in and you select your preferred language, so we're seeing here we're getting some German, so someone is listening to me now speaking in German. And so if you are, uh, now listening to this, so you probably need some headphones. So if you have some noise-canceling headphones, you can now listen into this talk in your preferred language. And so Gemini Live Translate is a really"
  },
  {
    "startMs": 497664,
    "text": "incredible model. It's near real-time translation, so as I'm speaking, as soon as the model is recognizing kind of the context of what I'm speaking about, uh, it will then start translating in real time to your desired language. So I see someone is listening in there. Do, do you get some audio out? Can you hear it? Yeah. Does it sound good? Fantastic. See. Um, so this is, this is really exciting because, you know, in a conference like this, obviously we"
  },
  {
    "startMs": 527604,
    "text": "have a very, very international audience. And so I can speak in my preferred language, so I could switch to German now, for example."
  },
  {
    "startMs": 548784,
    "text": "Live."
  },
  {
    "startMs": 551884,
    "text": "Power from, uh, Gemini Live Translate. Um, it is a, a really incredible model. Um, it automatically detects the language the speaker is speaking and then translates it in real time, uh, into our preferred language. Okay, we have, uh, a very international crowd here today. Uh, who's listening in, in, uh, Chinese? Are you listening in Chinese?"
  },
  {
    "startMs": 577244,
    "text": "Okay. Uh, I didn't understand that, but"
  },
  {
    "startMs": 583524,
    "text": "Okay. Um, so that is Gemini Live Translate. Um, if you wanna try it out, you can add it to your, um, applications now. It is available on the Gemini API, so do try that out. Uh, it is a really incredible model, and, uh, would love to hear kind of what you're, what you're building with that. Now, obviously we're looking at, uh, agents, right? Can your agents, uh... What's the, the title of this talk? Gemini made up"
  },
  {
    "startMs": 613484,
    "text": "this title, so I, uh, Can Your Agent Hear Me Now? I think that, that is the title. So our most recent real-time conversational model is called Gemini 3.1 Flash Live. Now, if you've talked to Gemini before in the Gemini app, that is exactly the model that you're speaking to. And so we can do a little demo here. So this demo was vibe-coded by Gemini in Google AI Studio. And if you remember the good old"
  },
  {
    "startMs": 643564,
    "text": "times, you know, when you were listening to radio, you know, not too long ago, you could phone up the, the, the radio host, and you could ask for a song, right? You could wish for a song. So it's the same idea here. We're using Gemini Live to be our radio DJ, and then we're giving Gemini Live an API, a function call to the Gemini API to then use Lyria 3, which is our music generation model, to then generate some music"
  },
  {
    "startMs": 673484,
    "text": "for us on demand. Okay. Let's, uh, try this out. We're, we're just going live demo to live demo here. Let's, let's see."
  },
  {
    "startMs": 682224,
    "text": "Well, hello there. You're live on the live jukebox right now. What absolute banger are we cooking up for you today, mate? To get this track properly sorted, I need a bit more of an idea. What genre are we thinking? Upbeat synth pop, smooth jazz, something with a bit more welly? And what's the vibe lyrics-wise? Give us some details."
  },
  {
    "startMs": 699944,
    "text": "Yeah, can we get, like, a really high energy reggaeton, please, about the AI Engineering World's Fair here in San Francisco?"
  },
  {
    "startMs": 710184,
    "text": "Now, that sounds like a proper belter, mate. High energy reggaeton about the AI Engineering World's Fair? Brilliant. Right, to make this absolutely top of the pops, any specific bits of tech or big names you want thrown in, or are we just going for that huge festival vibe?"
  },
  {
    "startMs": 725244,
    "text": "Yeah, just surprise me, please."
  },
  {
    "startMs": 729192,
    "text": "You got it, my friend. Prepare for a huge musical surprise. Generating that track right now. Mixing that reggaeton rhythm with the tech buzz for the world's fair. Watch this space, it's gonna blow your headphones off. Boom."
  },
  {
    "startMs": 740492,
    "text": "San Francisco, la bahía, con energía. AI Engineerin' fair, what a feeling, let's go. Siente el dembow, flow futurístico on the floor. Neural networks firing up, groundbreaking for sure. Clever algorithms dancing, la nueva sensación. Tech pioneers mingling, this is innovation. Bass line is pulsing heavy, can you feel the heat? Synth stabs electrifying, moving your feet. Celebrando la tecnología, no vamos a parar. En la ciudad con niebla, this is how we are. Feria de AI, la fiesta. ¡Dale! Feria de AI, la fiesta. ¡Wepa!"
  },
  {
    "startMs": 770132,
    "text": "Yeah, that, that was pretty cool. You went, uh, multilingual there as well. Nice one. Cheers."
  },
  {
    "startMs": 777052,
    "text": "Cheers, mate. Glad you like the flavor. Keep those big ideas coming. This jukebox never sleeps. What we spinning next, then? Feeling-"
  },
  {
    "startMs": 784832,
    "text": "All right, um, yes, you can try this out yourself. Uh, it is, is a, a demo that was built in Google AI Studio. Uh, so if you want, you can fork this in AI Studio and continue yourself. Um, I know this application might not necessarily be that useful for your business context, you know, if that's something you're looking to do. But I think the individual components are definitely useful. So if you're building real-time conversational agents,"
  },
  {
    "startMs": 815952,
    "text": "uh, you know, you want them to be multilingual. You want them to have capabilities, uh, tool calls. So in this example, we're, we're just giving the agent a tool call to generate music with Lyria 3, uh, on demand. Now, obviously, this could also just call out to your CRM and get some, you know, customer information, or it could create, you know, a new ticket on the back end. Um, so, you know, really what I want you to take away is kind of this real-time, multimodal conversational, uh,"
  },
  {
    "startMs": 845532,
    "text": "experience that you can build. And so the way that you can build with this model, so the, the Gemini 3.1 Flash Live real-time conversational model, is through the Live API. So the Live API is a low latency real-time voice and vision, uh, API, so you can build multimodal, uh, live real-time interactions. And so what this means is you have this stateful WebSocket API"
  },
  {
    "startMs": 875492,
    "text": "where you create a session, and then over this WebSocket, you can send, in real time, text, audio, and video frames to the model, and the model will then respond in real time back in audio chunks. And these audio chunks you can then stream, uh, in your application. Uh, and in addition to the audio, you also get the transcript of what was said. So if you've worked with voice agents before, this is a little bit different from"
  },
  {
    "startMs": 905432,
    "text": "kind of your cascading voice agent pipeline because this is a native audio model. So the model itself does not go through text, but rather the intelligence is baked into the audio model. So the audio model can understand the speech naturally, uh, and natively, and can then respond back in any of the 90 supported languages. So this is what allows us to, to switch very fluidly between different languages."
  },
  {
    "startMs": 936392,
    "text": "Um, so kind of the key features here, yeah, multilingual support, so 90, uh, different languages currently supported in the preview. Um, we can barge in. We can interrupt the model, right, with our DJ is talking a bit too lengthy. Uh, we, we can, we can barge in, interrupt it. We can give it, uh, the tools, and we have the audio transcription as well. Now, if you're building real-time conversational applications or real-time, um, multimodal applications, uh, you know, real-time"
  },
  {
    "startMs": 966372,
    "text": "media can be quite challenging to scale. So we have a great lineup of partners. So, you know, LiveKit, Pipecat, they're all actually here in the expo as well. I recommend you go visit them. They do some really, really cool stuff to allow you to get up and running quickly with the Gemini Live API. Okay. So if you're now, um, you know, you're like, \"Okay, this sounds great. I want this,\" um, the easiest way to try this out is through Google AI Studio."
  },
  {
    "startMs": 996932,
    "text": "So we spend a lot of money on this domain, ai.studio, so I do appreciate if you use that one, um, so we can get some ROI there. So ai.studio/live. You can try out the model. You can talk to it, and you can understand the capabilities that are available. So let's do that now. Uh, so we're just going to ai.dev. We also have ai.dev. Um, yeah, please do use that as well. That one was, was also not, not so cheap."
  },
  {
    "startMs": 1027072,
    "text": "Uh, there we go, ai.dev/live. And so here, uh, we can also try out the Live Translate model if you want to, uh, but we can here select our Gemini 3.1 Flash, uh, Live model. We can select a different voice. Uh, for example, I like Puck here. And then also we can enable grounding with Google Search. So obviously the model is trained on a lot of information, but it does not have access to real-time information, right?"
  },
  {
    "startMs": 1057212,
    "text": "And so by giving it access to Google Search, it can actually go off, and it's basically like a built-in function call. And if we asked it about today's weather or, you know, why Germany played so, uh, well in the World Cup, uh, we can, we can do that, and it can use Google to, to look that up. Okay, let's give this a try.  Hey, can you hear me?"
  },
  {
    "startMs": 1081412,
    "text": "Yes, I can hear you perfectly. How can I help you today?"
  },
  {
    "startMs": 1084852,
    "text": "Can you see me?"
  },
  {
    "startMs": 1087272,
    "text": "Yes, I can see you clearly. You're wearing a black shirt and a black hat, sitting in front of a blue background. Anything specific you wanted to show me?"
  },
  {
    "startMs": 1095252,
    "text": "Yeah. What does it say on my sweatshirt?"
  },
  {
    "startMs": 1098504,
    "text": "I can see the word Gemini printed in yellow letters. Anything else you'd like me to identify?"
  },
  {
    "startMs": 1103664,
    "text": "Yeah. Was, uh, heißt das denn auf Deutsch?"
  },
  {
    "startMs": 1106884,
    "text": "Das heißt auf Deutsch Zwillinge. Möchtest du noch etwas übersetzen?"
  },
  {
    "startMs": 1116084,
    "text": "Das bedeutet, ich kann dich sehen, Oma. Oh, ich sehe dich. Ähm, du siehst gut aus, Oma. Möchtest du noch etwas übersetzen lassen?"
  },
  {
    "startMs": 1124804,
    "text": "Ah, okay. I think my, uh,  my Mandarin didn't work out so well there. Uh, can you also speak Mandarin, please?"
  },
  {
    "startMs": 1133364,
    "text": "Yes, I can speak Mandarin too. What would you like to say now?"
  },
  {
    "startMs": 1152624,
    "text": "Oh.   All right. If you don't read Mandarin, I do apologize. I was just asking the model what it has eaten, uh, and it just said it doesn't need to eat because it's, it's a model. So, uh, yeah, there we go. All right. So this is the easiest way for you to get started with, uh, Gemini Live, ai.studio/live. If you want to learn more about building with Gemini and the Gemini Live"
  },
  {
    "startMs": 1182684,
    "text": "API, join me tomorrow morning in, uh, room 2009, I think. Check the schedule. It, it does change every once in a while. Uh, but other than that,"
  },
  {
    "startMs": 1195984,
    "text": ". Thank you. Cheers."
  }
]
