Sber open-sources KVAE-Audio tokenizer
Sber has released KVAE-Audio, an MIT-licensed 48 kHz continuous audio VAE that it says beats rival autoencoders on general-domain text-to-audio generation with far fewer latent channels.
Sber's Kandinsky team announced KVAE-Audio, a full-band (48 kHz) continuous tokenizer for sound. It adds audio to the lab's earlier image and video tokenizers, including the KVAE-2.0 video models. The code and inference are open under the MIT license, and the weights are on Hugging Face.
The team says the VAE is designed less for compression than for producing a latent space that diffusion models can learn from easily. The target uses are text-to-audio and joint text-to-video-plus-audio generation. The comparison baselines were the MMAudio VAE (44.1 kHz), which is widely used in open T2AV systems, Meta's DAC-based MovieGen Audio autoencoder, and SAME-L from Stable Audio 3.
KVAE-Audio is built on the Descript Audio Codec (DAC) and works directly on waveforms. MMAudio, by contrast, encodes mel-spectrograms and then needs a vocoder. Sber says this choice helps preserve phase information and removes the extra model. Its main changes to DAC are:
- a VAE bottleneck with reparameterization in place of residual vector quantization
- an extra encoder and decoder block, with strides of [2, 3, 4, 5, 8] giving 960x temporal compression
- 50 latent frames per second (20 ms each) instead of about 86
- 64 latent channels instead of 1,024
- self-attention in the bottleneck
- a proprietary regularization method meant to improve "diffusability"
Sber says the model gives the best general-domain text-to-audio quality among those tested while matching rivals on reconstruction. Reconstruction was scored with spectral, signal (SI-SDR, SDR, SNR) and PESQ-WB metrics on AudioSet, MUSDB18-HQ and EARS. The available excerpt does not include the full results tables, and no full technical report has been published.