High-quality Speech Coding with Sample RNN

By Janusz Klejsa, Per Hedelin, Cong Zhou, Roy Fejgin, Lars F. Villemoes. 2018

63 citations recorded by the source corpus.

We provide a speech coding scheme employing a generative model based on SampleRNN that, while operating at significantly lower bitrates, matches or surpasses the perceptual quality of state-of-the-art classic wide-band codecs. Moreover, it is demonstrated that the proposed scheme can provide a meaningful rate-distortion trade-off without retraining. We evaluate the proposed scheme in a series of listening tests and discuss limitations of the approach.

Subfields: Computer Vision and Pattern Recognition, Efficiency & Compression, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jan Skoglund
2. Jean-Marc Valin
3. Andrew Hines
4. Yang Ai
5. Zhenhua Ling
6. Alejandro Luebs
7. Yan Lu
8. Xue Jiang

In Efficiency & Compression

1. Jan Skoglund
2. Jean-Marc Valin
3. Mi Suk Lee
4. Jongmo Sung
5. Kai Zhen
6. Xiulian Peng
7. Minje Kim

In Speech & Audio

1. Jan Skoglund
2. Jean-Marc Valin
3. Minje Kim
4. Wootaek Lim
5. Haici Yang
6. Zhenhua Ling
7. Yang Ai
8. Alejandro Luebs

In Signal Processing

1. Jan Skoglund
2. Jean-Marc Valin
3. Minje Kim
4. Michael Chinen
5. Xue Jiang
6. Yan Lu
7. Xiulian Peng
8. Wootaek Lim

Citations in the corpus

• Audiodec: An Open-Source Streaming High-Fidelity Neural Audio Codec (2023, neutral): “The second category is the vocoder-based codec, which directly reconstructs audio signals using neural vocoders conditioned on the codes from other parametric coders [16] or quantized acoustic features [17–19].”
• Generative Speech Coding with Predictive Variance Regularization (2021, background): “Hence, despite extensive research, e.g., [8–12] generative synthesis based speech coding has not yet seen major practical applications.”
• ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric (2020, background): “There are other DNN-based generative models, including SampleRNN [11] and WaveGlow [12], with promising results that suggest that this trend will continue.”
• End-to-End Neural Speech Coding for Real-Time Communications (2022, neutral): “Existing neural audio codecs could mainly be grouped into two categories, generative de-coder models [1, 2, 3, 4, 5, 6] and end-to-end neural audio coding [7, 8, 9, 10].”
• Improving Opus Low Bit Rate Quality with Neural Speech Synthesis (2019, methodology): “ctical for a real-time implementation. Replacing the dilated convolutional networks with recurrent networks improved memory efficiency in SampleRNN [10], which has shown to be useful for speech coding [11]. WaveRNN [12] also demonstrated possibilities for synthesizing at lower complexities compared t”
• A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet (2019, methodology): “As reported in [10], mismatches between the training and testing database can cause a significant difference in the output quality.”
• Psychoacoustic Calibration of Loss Functions for Efficient End-to-End Neural Audio Coding (2020, background): “Recent efforts on deep neural network-based speech coding systems have made substantial progress on the coding gain [4]–[6].”
• Knowledge-and-Data-Driven Amplitude Spectrum Prediction for Hierarchical Neural Vocoders (2020, background): “as voice conversion [17, 18], bandwidth extension [19] and speech coding [20].”
• Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding (2022, methodology): “Their encoding parts are relatively simplified, relying on existing Codec 2 codes [5] as in the original WaveNet-based speech codec [4] or the dimension-reduced frequency-domain speech representations, e.g., cepstrum features [6, 7], and LPC analysis [8].”
• A Streamwise Gan Vocoder for Wideband Speech Coding at Very Low Bit Rate (2021, unknown): “The complexity issue was partially tackled with a different approach in [13].”
• Source Coding of Audio Signals with a Generative Model (2020, methodology): “We use a model similar to the one described in [2].”
• DRED: Deep REDundancy Coding of Speech Using a Rate-Distortion-Optimized Variational Autoencoder (2022, background): “In parallel, neural speech codecs [2], [3], [4] have also demonstrated their ability to efficiently transmit speech with sufficient quality at very low bitrates and we believe they can play a part in improving how we address packet loss.”
• Robust Low Rate Speech Coding Based on Cloned Networks and Wavenet (2020, methodology): “[19] is based on SampleRNN, operating at 6.”
• Cross-Scale Vector Quantization for Scalable Neural Speech Coding (2022, background): “Some researchers leverage advances in speech synthesizing with generative models [1, 2, 3, 4, 5, 6, 7, 8], such as WaveNet [5], its variants WaveGRU in Lyra [6], LPCNet [7] and SampleRNN [8].”