Counterpoint by Convolution

By Cheng-Zhi Anna Huang, Tim Cooijmans, Adam Roberts, Aaron C. Courville, Douglas Eck. 2019

166 citations recorded by the source corpus.

Machine learning models of music typically break up the task of composition into a chronological process, composing a piece of music in a single pass from beginning to end. On the contrary, human composers write music in a nonlinear fashion, scribbling motifs here and there, often revisiting choices previously made. In order to better approximate this process, we train a convolutional neural network to complete partial musical scores, and explore the use of blocked Gibbs sampling as an analogue to rewriting. Neither the model nor the generative procedure are tied to a particular causal direction of composition. Our model is an instance of orderless NADE (Uria et al., 2014), which allows more direct ancestral sampling. However, we find that Gibbs sampling greatly improves sample quality, which we demonstrate to be due to some conditional distributions being poorly modeled. Moreover, we show that even the cheap approximate blocked Gibbs procedure from Yao et al. (2014) yields better samples than ancestral sampling, based on both log-likelihood and human evaluation.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Multimodal Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Multimodal Learning

In Signal Processing

Citations in the corpus

Music Transformer: Generating Music with Long-Term Structure (2018, methodology): “B ACH C HORALES J.S. Bach chorales is a canonical dataset used for evaluating generative models for music 3 (e.g., Allan & Williams, 2005; Boulanger-Lewandowski et al., 2012; Liang, 2016; Hadjeres et al., 2016; Huang et al., 2017).”
Jukebox: A Generative Model for Music (2020, methodology): “More recent data-driven approaches include DeepBach (Hadjeres et al., 2017) and Coconet (Huang et al., 2017) which use Gibbs sampling to produce notes in the style of Bach chorals, MidiNet (Yang et al., 2017) and MuseGAN (Dong et al., 2018) which use generative adversarial networks, MusicVAE…”
Symbolic Music Generation with Diffusion Models (2021, neutral): “, 2020), and discrete symbolic music (Huang et al., 2017; 2018; Simon et al., 2018; Payne, 2019; Dinculescu et al., 2019; Donahue et al., 2019; Wang et al., 2020; Choi et al., 2020; Huang & Yang, 2020).”
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks (2023, background): “Recent work has shown the potential of large-scale models for music infilling (Huang et al., 2019a,b) and generation (Agostinelli et al., 2023; Copet et al., 2023; Ren et al., 2020).”
Learning a Latent Space of Multitrack Measures (2018, methodology): “closer to the MIDI standard itself. Some work in music generation has been focused on speciﬁc domains. One such popular domain for polyphonic music generation is Bach chorales: DeepBach [16], CoCoNet [18], and BachBot [24] are all different generative models for polyphonic Bach chorales that can res”
Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints (2016, background): “In line with a global surge in deep learning and neural network modeling over the past decade, several studies address the task of music modeling as a form of sequence learning, in which musical pieces are formulated as a time series of musical events, using state of the art sequence models such as…”
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models (2020, background): “Examples include DeepBach [24] and Coconet [27], both trained on four-part Bach Chorales.”
Fair Generative Modeling via Weak Supervision (2019, background): “, 2017), and a large number of creative applications such Coconet used for designing the “first AI-powered Google Doodle” (Huang et al., 2017).”
PIANOTREE VAE: Structured Representation Learning for Polyphonic Music (2020, background): “Some representative systems belonging to this category include DeepBach [16], XiaoIce [17], and Coconet [18].”
Human-AI Co-Creation in Songwriting (2020, methodology): “ the output at each intermediate stage before passing content into the next model. Joint modeling To generate multiple parts together, several teams adopted models such as MusicVAE trio [57], Coconet [35], MusicAutobot [62] or Transformers that are trained to jointly model multiple instrumental part”
An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation (2018, background): “Figure 1 shows how a pianoroll (left) can be represented as a grid (right), following [14].”
Encoding Musical Style with Transformer Autoencoders (2019, methodology): “Huang et al. (2019b) propose a novel algorithm for implementing relative self-attention that is signiﬁcantly more memory-efﬁcient, enabling the model to generate musical sequences over a minute in length.”
LakhNES: Improving Multi-instrumental Music Generation with Cross-domain Pre-training (2019, background): “Other research focuses on the multi-instrumental setting and seeks to provide systems which can harmonize with human-composed material [17–20].”