Improving Opus Low Bit Rate Quality with Neural Speech Synthesis

By Jan Skoglund, Jean-Marc Valin. 2019

41 citations recorded by the source corpus.

The voice mode of the Opus audio coder can compress wideband speech at bit rates ranging from 6 kb/s to 40 kb/s. However, Opus is at its core a waveform matching coder, and as the rate drops below 10 kb/s, quality degrades quickly. As the rate reduces even further, parametric coders tend to perform better than waveform coders. In this paper we propose a backward-compatible way of improving low bit rate Opus quality by re-synthesizing speech from the decoded parameters. We compare two different neural generative models, WaveNet and LPCNet. WaveNet is a powerful, high-complexity, and high-latency architecture that is not feasible for a practical system, yet provides a best known achievable quality with generative models. LPCNet is a low-complexity, low-latency RNN-based generative model, and practically implementable on mobile phones. We apply these systems with parameters from Opus coded at 6 kb/s as conditioning features for the generative models. A listening test shows that for the same 6 kb/s Opus bit stream, synthesized speech using LPCNet clearly outperforms the output of the standard Opus decoder. This opens up ways to improve the decoding quality of existing speech and audio waveform coders without breaking compatibility.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

High Fidelity Neural Audio Compression (2022, methodology): “Gârbacea et al. (2019) conditioned a WaveNet based model on discrete units obtained from a VQ-VAE (Van Den Oord et al., 2017; Razavi et al., 2019) model, while Skoglund & Valin (2019) tried feeding the Opus codec (Valin et al., 2012) to a WaveNet to further improve its perceptual quality.”
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “In [34] a WaveNet model was conditioned on discrete units obtained from a VQ-VAE model, while in [35] the Opus codec [36] was fed to WaveNet.”
Audio Codec Enhancement with Generative Adversarial Networks (2020, methodology): “In [13], a backward-compatible method was proposed for improving the speech quality of the Opus codec.”
End-to-End Neural Speech Coding for Real-Time Communications (2022, neutral): “Existing neural audio codecs could mainly be grouped into two categories, generative de-coder models [1, 2, 3, 4, 5, 6] and end-to-end neural audio coding [7, 8, 9, 10].”
Cross-Scale Vector Quantization for Scalable Neural Speech Coding (2022, background): “Some researchers leverage advances in speech synthesizing with generative models [1, 2, 3, 4, 5, 6, 7, 8], such as WaveNet [5], its variants WaveGRU in Lyra [6], LPCNet [7] and SampleRNN [8].”