Sber Publishes KVAE-Audio with Open Source Code

Sber has made an algorithm for audio data compression publicly available. The tool processes sound at a frequency of 48 kHz and compresses it by 960 times in time. The resulting latent space is limited to 64 channels. The compact representation size allows for significantly faster training of generative models than if they were trained on raw audio signals. When developing such solutions, a precise balance is always required between the quality of restoring the original sound and generating new ones. Engineers implemented a custom regularization technique to address the typical problem of strong bias in algorithms towards reconstruction. Thanks to this approach, the architecture outperformed Sony's MMAudio tokenizer in all measurable metrics. The model also surpassed Meta's DACVAE and Stability AI's SAME-L in terms of generation while maintaining a similar level of restoration with a radically smaller number of parameters. The project is a logical continuation of the KVAE family. Previously, solutions for image and video compression were published in this line. The code and weights are available on GitHub and HuggingFace under the MIT open-source license.