Valeria Wu Fon is a product leader whose work on Gemini’s speech-to-speech models connects conversational AI with the practical demands of getting things done by voice. Her focus extends beyond natural-sounding answers: spoken agents need to follow instructions, use tools accurately, understand their surroundings, and respond quickly enough to keep a conversation moving.
From education finance to spoken AI
Wu Fon studied Symbolic Systems at Stanford and worked at Google and Credit Suisse earlier in her career. She subsequently held a product-lead role at Cometa, an education-finance company. That chapter placed her work at the intersection of education, financial services, and product development, before her later role leading product for Gemini’s speech-to-speech model at Google DeepMind.
Her Gemini work connects a shared speech-model foundation to several kinds of interaction. Gemini Live and Search Live bring spoken assistance into consumer products, while the Gemini Live API lets developers build their own conversational applications. The underlying ambition is a model versatile enough to support these different experiences through prompting: helping someone understand what their camera sees, answering a question, translating a conversation, or calling a tool to retrieve information and take action.
Wu Fon co-authored the September 2025 Gemini Live API announcement, connecting model improvements to practical developer needs, including more reliable adherence to tool schemas and better interruption handling. A spoken request becomes useful action only when the assistant selects an appropriate function and supplies arguments that the surrounding software can execute. Conversational fluency has to survive that transition from talking to doing.
Her product work also extends to Gemini 3.1 Flash Live, introduced in March 2026 with improvements in tonal understanding and conversational context. These capabilities address another part of the experience: interpreting how someone speaks and maintaining enough continuity to respond appropriately as an exchange develops. Across consumer products and developer applications, her concerns encompass both successful task execution and the timing and context that make assistance feel conversational.
What useful voice agents require
Wu Fon’s product priorities combine conversational timing, task intelligence, and multimodal interaction. More reasoning can improve an answer or a tool decision, yet the extra computation can delay the first spoken response. Her approach to speech-to-speech product development treats this as a continuing tradeoff: improve what the model can accomplish while preserving the responsiveness people expect when speaking.
Reliable action through voice. Wu Fon places instruction following, reasoning, and task completion alongside conversation quality. A roadside-assistance example makes the requirements concrete: an agent needs to capture registration plates and postcodes accurately, retrieve policy information, and keep helping while traffic passes nearby. Small recognition errors can prevent the system from finding the right record, even when its replies sound convincing.
Proactive audio in everyday environments. Wu Fon emphasizes that an assistant must decide which sounds warrant a response. Nearby conversation or background noise should not automatically interrupt its answer. Spoken interactions happen on trains, during walks, and beside roads; selective attention allows the agent to continue a useful exchange amid incidental sound rather than depend on a quiet room.
Multilingual interaction with context. Language support belongs within her core product ambition. Streaming translation should begin while someone is still speaking, allowing a conversation to proceed without rigid pauses between turns. Localization also requires judgment: an assistant can use regional Spanish while leaving a familiar design term in English when that is how people commonly express it.
Multimodality that helps complete the task. A camera view can spare a user from describing a sofa in detail; spoken questions can guide a visual search, and retrieved search cards can provide something concrete to inspect afterward. Wu Fon’s examples connect incoming audio and video with useful outputs and tool calls. The purpose is to give the assistant the context it needs and choose a response format that helps the user continue.
The model foundation spans audio, video, and text. Wu Fon’s product ambition is to make that breadth usable in a single assistant that can move between translation, information seeking, taking action, brainstorming, and less structured conversation. Each mode places different demands on the system, but the person speaking should be able to move between them without losing the task or the rhythm of the exchange.
Valeria Wu Fon and Tom Ouyang explain how joint audio, video, and text training supports versatile voice agents—and why reasoning, response timing, language use, and visual output must work together.
End-to-end speech recognition simplified audio-to-text modeling, but translation, responses, speaker interpretation, and visual context still required additional system work.
Interleaved audio, video, and text pre-training teaches cross-modal relationships that a shared model can draw on for translation, visual questions, and speech generation.
Useful voice behavior includes selective localization and selective attention: keep familiar borrowed terms when appropriate, and avoid treating every background sound as a conversational interruption.
The longer-term goal is a single promptable model that can switch among translation, task execution, brainstorming, and informal conversation while coordinating multimodal input and output.
▶ Watch ↗