Synthesizing Audio with Generative Adversarial Networks

By Chris Donahue, Julian McAuley, Miller S. Puckette. 2018

165 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

WaveGrad: Estimating Gradients for Waveform Generation (2020, background): “The upsampling block (UBlock) of WaveGrad follows the GAN-TTS generator, with a minor difference that no BatchNorm is used.”
The challenge of realistic music generation: modelling raw audio at scale (2018, background): “Previous work on music modelling in the raw audio domain [10, 13, 31, 43] has shown that capturing local structure (such as timbre) is feasible, but capturing higher-level structure has proven difficult, even for models that should be able to do so in theory because their receptive fields are large ”
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis (2020, background): “This problem has been addressed in the previous work (Donahue et al., 2018) by increasing receptive ﬁelds of the generator and discriminator.”
MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis (2019, methodology): “We solve this problem by carefully choosing the kernel-size and stride for our deconvolutional layers as a simpler alternative to PhaseShufﬂe layer introduced in Donahue et al. (2018b).”
WaveFlow: A Compact Flow-based Model for Raw Audio (2019, background): “Non-autoregressive convo- lutional architectures can do speedy synthesis and easily capture the long-range structure in the data (Radford et al., 2015; Brock et al., 2018), but it could produce spurious high-frequency components which will hurt the audio fidelity (e.g., Donahue et al., 2018).”
MelNet: A Generative Model for Audio in the Frequency Domain (2019, background): “Recent works have used generative adversarial networks (GANs) [14] to model both waveforms and spectral representations [12, 13].”
LOGAN: Latent Optimisation for Generative Adversarial Networks (2019, background): “…including representation learning and inference (Donahue et al., 2017; Dumoulin et al., 2017), text generation (Zhang et al., 2019), style learning (Zhu et al., 2017; Karras et al., 2019), audio generation (Donahue et al., 2018) and video generation (Vondrick et al., 2016; Clark et al., 2019).”
Generative Adversarial Networks: A Survey and Taxonomy (2019, methodology): “GANs have been applied to various domains such as computer vision [7]–[14], natural language processing [15]–[18], time series synthesis [19]–[23], semantic segmentation [24]–[28] etc. GANs belong to the family of generative models.”
Conditional WaveGAN (2018, unknown): “One of them is WaveGAN [5], which trains a generative model in an unsupervised setting.”
Bandwidth Extension on Raw Audio via Generative Adversarial Networks (2019, background): “Notably, the authors of [7] show that non-trivial modifications of GAN architectures are required to generative diverse and plausible audio outputs.”
EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals (2018, background): “GANs have mainly been developed and applied to the generation of images and only a few studies investigating time series were conducted; recently they showed promising re-sults for the generation of artiﬁcial audio (Donahue et al., 2018).”
Synthetic-Neuroscore: Using a neuro-AI interface for evaluating generative adversarial networks (2019, methodology): “Specifically, GANs have been widely applied to various domains such as computer vision [3], natural language processing [4], speech synthesis [5] and time series synthesis [6].”
Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption (2020, methodology): “As another application of adversarial training, GAN [13] has been widely used in multiple machine learning tasks, such as computer vision [10, 20, 33], natural language processing [12, 36, 8] and time series synthesis [9, 11].”
Deep Learning for Audio Signal Processing (2019, background): “Despite the success of GANs [55] for image synthesis, their use in the audio domain has been limited.”
Speech Enhancement with a Generative Adversarial Network (2019, background): “Phase shift (Donahue et al., 2018) can reduce high-frequency artifacts in the outputs ofG, while spectral normalization (Miyato et al.”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, background): “These models have a variety of applications in the real-world, such as generation of text-to-speech [53], text-to-image [54], and training data for medical imaging [55].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “Additionally, GANs have made achievements in sequential data tasks, such as those involving natural language, music, speech, voice [268], [269], and time series data [270], [271], [272], [273].”
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow (2018, background): “A particular instantiation is generative adversarial networks, which can be used for high-ﬁdelity generation of images (Goodfellow et al., 2014; Karras et al., 2017) and other high-dimensional data (Vondrick et al., 2016; Xie et al., 2018; Donahue et al., 2018).”