Kandinsky 6.0 Video adds synchronized audio
Kandinsky Lab has released Kandinsky 6.0 Video, an MIT-licensed model family of up to 29B parameters that generates video with synchronized audio, adding to the small pool of open joint audio-video generators.
The Kandinsky team described the release in an arXiv paper. Kandinsky 6.0 Video is a family of diffusion models that generate video and audio together from text. It comes in two sizes: Lite, with 3 billion parameters, and Pro, with 29 billion.
Both models produce 5-second clips with synchronized 44 kHz audio, including lip-sync. They work from either a text prompt (text-to-audio-video) or a starting image (image-to-audio-video). A built-in super-resolution model raises the output to Full HD (1920×1080).
The design builds on Kandinsky 5.0's video generation. It uses a dual-stream CrossDiT architecture that links the pretrained video stream to a newly trained audio stream through bidirectional cross-attention, so the two stay aligned in timing and meaning. The audio stream was first trained from scratch on large audio datasets. Both streams were then trained together on paired audio-video data in a way meant to keep the quality of each one. After pretraining came supervised fine-tuning, reinforcement-learning-based post-training, and distillation.
The evidence for quality is limited. In side-by-side human evaluations, the authors say Kandinsky 6.0 Video Pro "clearly outperforms" Kandinsky 5.0 Video Pro and "remains competitive" with leading audio-video generation models, especially on speech quality. The abstract does not name the models it was compared against or give scores.
The code, model checkpoints, and integration with the Hugging Face diffusers library are released under the MIT license.