C-RNN-GAN: Continuous recurrent neural networks with adversarial training

By Olof Mogren. arXiv.org, 2016

588 citations recorded by the source corpus.

Generative adversarial networks have been proposed as a way of efficiently training deep generative neural networks. We propose a generative adversarial model that works on continuous sequential data, and apply it by training it on a collection of classical music. We conclude that it generates music that sounds better and better as the model is trained, report statistics on generated music, and let the reader judge the quality by downloading the generated songs.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f01efc7e1515243b8b80a3b2230b6bc6f6247ef3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Signal Processing

Citations in the corpus

MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment (2017, methodology): “C-RNN-GAN (Mogren 2016) generated polyphonic music as a series of note events by introducing some ordering of notes and using RNNs in both the generator and the discriminator.”
Differentially Private Generative Adversarial Network (2018, background): “By combining the complexity of deep neural networks and game theory, the Generative Adversarial Network (GAN) [16] and its variants have demonstrated impressive performance in modeling the underlying data distribution, generating high quality “fake" samples that are hard to be differentiated from re”
eCommerceGAN : A Generative Adversarial Network for E-commerce (2018, background): “When GANs are used with image data or music data, it is possible to view the generated images or listen to the generated musics and qualitatively check if the GAN has effectively learned the real data distribution [4, 12, 15].”
Electrocardiogram generation with a bidirectional LSTM-CNN generative adversarial network (2019, methodology): “Mogren et al . proposed a method called C-RNN-GAN 35 and applied it on a set of classic music.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “1 Music C-RNN-GAN [78] models both generator and discriminator as an RNN with Long-short Term Memory (LSTM) [46] which is widely used in managing sequential data.”
PIANOTREE VAE: Structured Representation Learning for Polyphonic Music (2020, background): “Similarly, we see the note-based approaches [27, 28], in which polyphonic music is represented as a sequence of note tuples, as an alternative to the MIDI-like meth-ods.”
Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs (2017, unknown): “We are aware of only one preliminary work using GANs to generate continuous-valued sequences, which aims to produce polyphonic music using a GAN with LSTM generator and discriminator [19].”
Deep Learning Techniques for Music Generation - A Survey (2017, methodology): “The objective of Mogren’s C-RNN-GAN [123] system is the generation of single voice polyphonic music.”
How Generative Adversarial Networks and Their Variants Work (2017, methodology): “meidentitywhileattainingdiﬀerentposesfromtwogenerators. 4.2Sequentialdatageneration Besidesimages,GANisalsoapplicabletogeneratingsequentialdatasuchasmusic,language andevenvideos. 4.2.1Music C-RNN-GAN [71] models both generator and discriminator as RNN with Long-short Term Memory(LSTM) which is widel”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “Music: GANs have been used for generating music such as continuous RNN-GAN (C-RNN-GAN) [91], ObjectReinforced GAN (ORGAN) [92], and SeqGAN [93], [94].”
Data Augmentation techniques in time series domain: a survey and taxonomy (2022, methodology): “Then they compared the distribution of the data in the two-dimensional space for TimeGAN[55], Recurrent Conditional GAN (GAN)[56], Continuous recurrent GAN (GAN)[58], T-Forcing[59], WaveNet[60] and WaveGAN[61].”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, methodology): “Music generation: C-RNN-GAN [146], Seq-GAN [148], OR-GAN [149]”
MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer (2018, background): “However, Mogren [25], Yang et al. [34] and Dong et al. [10] have recently shown the efﬁcacy of CNN-based GANs for music composition.”
Learning Deep Hidden Nonlinear Dynamics from Aggregate Data (2018, methodology): “As for the function h (in both ﬁltering and smoothing), we use a recurrent neural network (RNN) to model it, similar to (Mogren, 2016).”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, background): “Generative adversarial networks (GAN) was a progress milestone in DL (Goodfellow et al., 2014; Mogren, 2016) motivated by the need to model high-dimensional, multimodal distributions.”
Applications of Generative Adversarial Networks (GANs): An Updated Review (2019, methodology): “A simple and direct approach is continuous RNN-GAN (C-RNN-GAN) [82], where it models both the generator and discriminator as an RNN with long-short term memory (LSTM) [40], directly extracting whole sequences of music.”
A Note on the Inception Score (2018, background): “…natural images [Karras et al., 2017], in addition to applications in style transfer [Zhu et al., 2017,Isola et al., 2016], image super-resolution [Ledig et al., 2016], natural language generation [Guo et al., 2017], music generation [Mogren, 2016], medical data generation [Esteban et al., 2017],…”