A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet

By Jean-Marc Valin, Jan Skoglund. 2019

84 citations recorded by the source corpus.

Neural speech synthesis algorithms are a promising new approach for coding speech at very low bitrate. They have so far demonstrated quality that far exceeds traditional vocoders, at the cost of very high complexity. In this work, we present a low-bitrate neural vocoder based on the LPCNet model. The use of linear prediction and sparse recurrent networks makes it possible to achieve real-time operation on general-purpose hardware. We demonstrate that LPCNet operating at 1.6 kb/s achieves significantly higher quality than MELP and that uncompressed LPCNet can exceed the quality of a waveform codec operating at low bitrate. This opens the way for new codec designs based on neural synthesis models.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

High Fidelity Neural Audio Compression (2022, methodology): “Neural based audio codecs have been recently proposed and demonstrated promising results (Kleijn et al., 2018; Valin & Skoglund, 2019b; Lim et al., 2020; Kleijn et al., 2021; Zeghidour et al., 2021; Omran et al., 2022; Lin et al., 2022; Jayashankar et al., 2022; Li et al.; Jiang et al., 2022),…”
A Survey on Neural Speech Synthesis (2021, methodology): “LPCNet [357, 358] introduces conventional digital signal processing into neural networks, and uses linear prediction coefficients to calculate the next waveform point while leveraging a lightweight RNN to compute the residual.”
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “In [32] an LPCNet [33] vocoder was conditioned on hand-crafted features and a uniform quantizer.”
Enhancing into the Codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders (2021, neutral): “6 kb/s [2, 3].”
Improving Opus Low Bit Rate Quality with Neural Speech Synthesis (2019, neutral): “3 of [27].”
Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech (2020, methodology): “AR models have been recently modified to speed up their inference [4, 5, 6].”
Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, background): “Previous work mostly focused on the optimization of vocoder for edge deployment [3-5], while less attention has been paid to the acoustic model [6].”
Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding (2022, methodology): “In particular, we are based on the efficient LPCNetbased speech coding framework [7], and our analysis and model training mainly focuses on the cepstral coefficients.”
An Efficient Subband Linear Prediction for LPCNet-Based Neural Synthesis (2020, unknown): “As a comparison, the total complexity of fullband WaveRNN and WaveNet vocoder is 10 GFLOPS and 100 GFLOPS[26] approximately.”
Source Coding of Audio Signals with a Generative Model (2020, methodology): “Deep generative models have been successfully used for speech coding [1–5], providing a significant improvement to the perceptual quality–bitrate tradeoff.”
A Streamwise Gan Vocoder for Wideband Speech Coding at Very Low Bit Rate (2021, unknown): “• We demonstrate that SSMGAN is able to generate high-quality speech even when conditioned with a parametric and highly compressed representation provided by the encoder of LPCNet [2], which delivers a 1 .”
DRED: Deep REDundancy Coding of Speech Using a Rate-Distortion-Optimized Variational Autoencoder (2022, background): “In parallel, neural speech codecs [2], [3], [4] have also demonstrated their ability to efficiently transmit speech with sufficient quality at very low bitrates and we believe they can play a part in improving how we address packet loss.”
Harp-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Coding (2021, unknown): “A more recent approach proposed to use LPCNet [10], achieving as low as 1.6kbps [11].”
Robust Low Rate Speech Coding Based on Cloned Networks and Wavenet (2020, background): “4 to 8 kb/s, and matches or exceeds AMR-WB quality at 23 kb/s. [20] uses linear prediction to facilitate reduced computational effort, and operates at 1 .”