Generative media and video AI
Runway
Runway develops generative models for creating and editing visual media, serving people making images and videos and developers building media applications. Gen-4.5 generates video from text or an image, while Aleph 2.0 edits existing video with keyframe-guided control. These offer two ways to work with moving images: generating footage from a description or starting image, and directing changes to footage that already exists.
Its research roots include work behind Stable Diffusion. In a 2022 interview with Patrick Esser, then a principal research scientist at Runway, Esser traced that work to research with Robin Rombach and other CompVis collaborators on representations useful for image synthesis. Their approach modeled learned image representations rather than treating images only as grids of pixels; scaling the latent diffusion work led to Stable Diffusion. Esser also described extending image-generation research into video synthesis and editing.
Runway’s model API gives developers access to its own models alongside those from other labs through one interface. The catalog spans video, image, audio and real-time models, allowing developers to combine media capabilities and switch models within an application.
1 talk
Newest firstLoading more talks and speakers…
1 speaker at AIE
Affiliations reflect their AIE event listings and recorded appearances, not necessarily current employment.
