Runway details Gen-2 video generation model
Runway's Gen-2 generates new video from text, images or existing clips, and the company says users prefer its output to existing methods in user studies.
Runway Research describes Gen-2 as a multimodal model that synthesizes new video "realistically and consistently." It works two ways: from a text prompt alone (text to video), or by applying the composition and style of an image or prompt to the structure of a source video (video to video).
The research page lists eight modes:
- Text to Video: generates video from a text prompt alone.
- Text + Image to Video: combines a driving image with a text prompt.
- Image to Video: generates video from a single driving image, which Runway calls "Variations Mode."
- Stylization: applies the style of an image or prompt to every frame of a video.
- Storyboard: turns mockups into stylized, animated renders.
- Mask: stylizes selected regions of a video and leaves the rest unchanged.
- Render: turns untextured 3D renders into realistic footage using an input image or prompt.
- Customization: adapts the model for higher-fidelity results.
Runway says that in user studies, people preferred Gen-2's results to existing methods for image-to-image and video-to-video translation. The page doesn't give the study design, sample size, preference rates or which systems Gen-2 was compared against.
It also leaves out the model architecture, training data, output resolution, clip length, pricing and availability. Runway presents Gen-2 as another step toward its stated goal of building "multimodal AI systems that will enable new forms of creativity."