← All speakers

Bio, Work & Ideas

Neil Zeghidour

Conference affiliation: Gradium AI · 2026

On this page

Neil Zeghidour is co-founder and chief executive of Gradium, which builds speech models and infrastructure for real-time voice applications. His research includes Google’s SoundStream and AudioLM and Kyutai’s Moshi, a conversational model capable of listening and speaking simultaneously.

From audio compression to voice agents

Gradium’s launch announcement identifies Zeghidour’s research background at Meta and Google DeepMind.

He was the first listed author of SoundStream, a neural codec for compressing speech, music, and other audio. He subsequently co-authored AudioLM, which generates coherent speech and music using discrete audio representations.

He later helped create Kyutai, an open-research laboratory in Paris, and co-authored Moshi and Hibiki, a simultaneous speech-to-speech translation system. His talks highlight two central challenges: audio instruction data that teaches models to use vocal cues and multistream architectures that represent simultaneous conversation.

In September 2025, Zeghidour co-founded Gradium with Olivier Teboul, Laurent Mazaré, and Alexandre Défossez. The company launched publicly on December 2 with $70 million in seed funding and a focus on voice infrastructure, including streaming speech recognition, synthesis, and voice cloning.

What natural voice AI actually requires

  • Full-duplex speech: Human conversations include interruptions, overlapping voices, and brief acknowledgments. Moshi can listen and respond concurrently, avoiding the rigid conversational turns that cause many voice assistants to stumble.
  • Production-ready conversational agents: Natural speech alone cannot replace dependable tool use, safety controls, observability, and personalization. In his Europe talk, Zeghidour acknowledges that conventional speech-recognition, language-model, and speech-synthesis pipelines remain practical to inspect and deploy, and argues that integrated speech systems must match their reliability, intelligence, and personalization before replacing them.
  • Application-level latency: External searches and tool calls can delay conversations more severely than speech synthesis. His demonstrated approach keeps an assistant speaking naturally while information is retrieved, then incorporates the result without an awkward pause. It improves conversational continuity rather than speeding up the underlying retrieval.
  • Paralinguistic understanding: Hesitation, discomfort, tone, and other vocal signals convey information that transcripts discard. Zeghidour argues that speech models must be trained to recognize and act on these cues, not merely receive audio inputs.
  • Gradium Phonon: Gradium’s on-device text-to-speech model, presented by Zeghidour, runs on a smartphone CPU, making speech synthesis less dependent on cloud inference and allowing TTS processing to stay local. This does not establish that recognition, reasoning, tools, or an entire interaction remain on the device. His stated goal is practical consumer-scale voice AI without prohibitive synthesis API costs; licensing and other application costs still matter.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

A cloned voice can sound convincing while the conversation still fails. Neil Zeghidour traces the remaining problems through latency, overlapping speech, agent capabilities and on-device synthesis.

  • From a voice recording to a synthetic speaker
    0:26 ↗
  • A natural voice is only part of the interaction
    2:38 ↗
  • The latency budget extends beyond speech synthesis
    5:42 ↗
  • Speech-to-speech does not guarantee simultaneous conversation
    9:02 ↗
  • Moshi keeps listening while answering
    11:51 ↗
  • Retaining vocal cues is not the same as using them
    12:55 ↗
  • Always-on voice changes the economics
    15:23 ↗
  • Moving synthesis onto a smartphone CPU
    17:00 ↗

Key ideas

Scroll to read ↓

Neil Zeghidour traces voice interfaces from rigid assistants to tool-using agents, then shows why fast speech-to-speech still feels unnatural—and how audio tokens, simultaneous streams, and a hybrid voice-plus-text architecture could fix it.

  • Open-ended dialogue, external action, expressive speech, low latency, and simultaneous conversation are distinct capabilities; improving one does not guarantee the others.
    2:35 ↗
  • Speech-to-speech can remove the explicit text cascade, but a half-duplex system still alternates between listening and speaking. Backchannels therefore become unwanted turn changes.
    7:14 ↗
  • Neural codecs and multi-stream modeling solve different problems: codecs compress long waveforms into audio tokens, while two streams represent simultaneous speech and silence for both participants.
    10:16 ↗
  • Full duplex makes overlap resilient but does not guarantee good timing or strong reasoning. The early system could interrupt too often while remaining less capable than cascaded agents.
    12:46 ↗
  • The hybrid proposal gives a small full-duplex model responsibility for conversational flow and delegates difficult reasoning and tools to a replaceable text backend. Its success depends on reliable delegation and asynchronous coordination.
    16:15 ↗

References