Enhancing into the Codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders

By Jonah Casebeer, Vinjai Vale, Umut Isik, Jean-Marc Valin, Ritwik Giri, A. Krishnaswamy. 2021

28 citations recorded by the source corpus.

Audio codecs based on discretized neural autoencoders have recently been developed and shown to provide significantly higher compression levels for comparable quality speech out-put. However, these models are tightly coupled with speech content, and produce unintended outputs in noisy conditions. Based on VQ-VAE autoencoders with WaveRNN decoders, we develop compressor-enhancer encoders and accompanying decoders, and show that they operate well in noisy conditions. We also observe that a compressor-enhancer model performs better on clean speech inputs than a compressor model trained only on clean speech.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

SoundStream: An End-to-End Neural Audio Codec (2021, background): “The work in [41] trains a speech enhancement system with a quantized bottleneck.”
Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding (2022, methodology): “Various other methods have been developed to improve the quality of the features, regarding robustness [18, 19], scalability [20] and the variability issues [21].”
DRED: Deep REDundancy Coding of Speech Using a Rate-Distortion-Optimized Variational Autoencoder (2022, background): “Although VQ-VAE [15] has been a popular choice for deep speech coding [16], [17], in this work we avoid its large fixed-size codebooks and investigate other variational auto-encoders (VAE) [18].”
Cross-Scale Vector Quantization for Scalable Neural Speech Coding (2022, background): “On the other hand, some researchers propose end-to-end neural networks based on the vector-quantized variational autoencoder framework (VQ-VAE [9]) where the encoder, codebook and the decoder are learned in a joint fashion [10, 11, 12, 13, 14, 15].”