Bandwidth Extension on Raw Audio via Generative Adversarial Networks

By Sung Kim, Visvesh S. Sathe. 2019

33 citations recorded by the source corpus.

Neural network-based methods have recently demonstrated state-of-the-art results on image synthesis and super-resolution tasks, in particular by using variants of generative adversarial networks (GANs) with supervised feature losses. Nevertheless, previous feature loss formulations rely on the availability of large auxiliary classifier networks, and labeled datasets that enable such classifiers to be trained. Furthermore, there has been comparatively little work to explore the applicability of GAN-based methods to domains other than images and video. In this work we explore a GAN-based method for audio processing, and develop a convolutional neural network architecture to perform audio super-resolution. In addition to several new architectural building blocks for audio processing, a key component of our approach is the use of an autoencoder-based loss that enables training in the GAN framework, with feature losses derived from unlabeled data. We explore the impact of our architectural choices, and demonstrate significant improvements over previous works in terms of both objective and perceptual quality.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Oleg Rybakov
2. Marco Tagliasacchi
3. Yunpeng Li
4. Dominik Roblek
5. Victor Ungureanu

Citations in the corpus

• Real-Time Speech Frequency Bandwidth Extension (2020, methodology): “All these methods are trained by minimizing a reconstruction loss, typically the The design of SEANet is similar to [12], but we adopt the losses proposed in [13], in which the reconstruction loss is computed in the feature space of the discriminator, at different scales and at different layers.”
• NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling (2021, unknown): “There are several works reported that SNR is not effective in the upsampling task because it could not measure high-frequency generation [2, 5].”
• A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, unknown): “Compared to previous work with 16kHz as the target frequency [5, 25, 39, 46, 53, 55, 56, 84], NU-Wave [49] is the first to synthesize 48kHz waveforms”