A Streamwise Gan Vocoder for Wideband Speech Coding at Very Low Bit Rate

By Ahmed Mustafa, Jan Büthe, Srikanth Korse, Kishan Gupta, Guillaume Fuchs, N. Pia. 2021

23 citations recorded by the source corpus.

Recently, GAN vocoders have seen rapid progress in speech synthesis, starting to outperform autoregressive models in perceptual quality with much higher generation speed. However, autoregressive vocoders are still the common choice for neural generation of speech signals coded at very low bit rates. In this paper, we present a GAN vocoder which is able to generate wideband speech waveforms from parameters coded at 1.6 kbit/s. The proposed model is a modified version of the StyleMelGAN vocoder that can run in frame-by-frame manner, making it suitable for streaming applications. The experimental results show that the proposed model significantly outperforms prior autoregressive vocoders like LPC-Net for very low bit rate speech coding, with computational complexity of about 5 GMACs, providing a new state of the art in this domain. Moreover, this streamwise adversarial vocoder delivers quality competitive to advanced speech codecs such as EVS at 5.9 kbit/s on clean speech, which motivates further usage of feedforward fully-convolutional models for low bit rate speech coding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Lei He
3. Sheng Zhao
4. Yanqing Liu
5. Tao Qin
6. Tie-Yan Liu
7. Rui Xue
8. Frank K. Soong

In Computer Vision and Pattern Recognition

1. Yanqing Liu
2. Sheng Zhao
3. Xu Tan
4. Lei He

In Signal Processing

1. Xu Tan
2. Lei He
3. Sheng Zhao
4. Yanqing Liu
5. Tao Qin
6. Rui Xue
7. Tie-Yan Liu
8. Yi Ren

Citations in the corpus

• Audiodec: An Open-Source Streaming High-Fidelity Neural Audio Codec (2023, neutral): “The second category is the vocoder-based codec, which directly reconstructs audio signals using neural vocoders conditioned on the codes from other parametric coders [16] or quantized acoustic features [17–19].”
• DelightfulTTS 2: End-to-End Speech Synthesis with Adversarial Vector-Quantized Auto-Encoders (2022, background): “Specifically, it consists of a symmetric encoder-decoder network [10, 11, 12, 13] with skip-connections between bottom and top layers, and a multi-stage vector quantizer [14] in the middle as feature bottleneck.”