Audio Codec Enhancement with Generative Adversarial Networks

By Arijit Biswas, D. Jia. 2020

46 citations recorded by the source corpus.

Audio codecs are typically transform-domain based and efficiently code stationary audio signals, but they struggle with speech and signals containing dense transient events such as applause. Specifically, with these two classes of signals as examples, we demonstrate a technique for restoring audio from coding noise based on generative adversarial networks (GAN). A primary advantage of the proposed GAN-based coded audio enhancer is that the method operates end-to-end directly on decoded audio samples, eliminating the need to design any manually-crafted frontend. Furthermore, the enhancement approach described in this paper can improve the sound quality of low-bit rate coded audio without any modifications to the existent standard-compliant encoders. Subjective tests illustrate that the proposed enhancer improves the quality of speech and difficult to code applause excerpts significantly.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Speech & Audio

In Signal Processing

Citations in the corpus

SoundStream: An End-to-End Neural Audio Codec (2021, methodology): “Audio enhancement – Deep neural networks have been applied to different audio enhancement tasks, ranging from denoising [17]–[21] to dereverberation [22], [23], lossy coding denoising [2] and frequency bandwidth extension [1], [24].”
High Quality Audio Coding with Mdctnet (2022, methodology): “For general audio coding, the main tracks of research have been to improve the outputs of traditional codecs [5-10] or to train a neural net end-to-end as an autoencoder for regression according to a perceptually motivated loss function [11].”
Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding (2022, methodology): “To mitigate the issue, some efforts apply more complex models in the encoder to improve the quality of the features [9, 10] or use generative models for post-processing [11, 12, 13] at the end of the existing codec to facilitate signal restoration.”