Learning a Latent Space of Multitrack Measures

By Ian Simon, Adam Roberts, Colin Raffel, Jesse Engel, Curtis Hawthorne, Douglas Eck. 2018

55 citations recorded by the source corpus.

Discovering and exploring the underlying structure of multi-instrumental music using learning-based approaches remains an open problem. We extend the recent MusicVAE model to represent multitrack polyphonic measures as vectors in a latent space. Our approach enables several useful operations such as generating plausible measures from scratch, interpolating between measures in a musically meaningful way, and manipulating specific musical attributes. We also introduce chord conditioning, which allows all of these operations to be performed while keeping harmony fixed, and allows chords to be changed while maintaining musical "style". By generating a sequence of measures over a predefined chord progression, our model can produce music with convincing long-term structure. We demonstrate that our latent space model makes it possible to intuitively control and generate musical sequences with rich instrumentation (see this https URL for generated audio).

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Symbolic Music Generation with Diffusion Models (2021, neutral): “, 2020), and discrete symbolic music (Huang et al., 2017; 2018; Simon et al., 2018; Payne, 2019; Dinculescu et al., 2019; Donahue et al., 2019; Wang et al., 2020; Choi et al., 2020; Huang & Yang, 2020).”
PIANOTREE VAE: Structured Representation Learning for Polyphonic Music (2020, methodology): “Models MIDI-like event sequence is ﬁrst used in deep music generation in performanceRNN [24] and Multi-track Music-VAE [9], and then broadly applied in transformer-based generation [19, 25, 26].”
Human-AI Co-Creation in Songwriting (2020, background): “The popular song form has also been an active area of research to tackle modeling challenges such as hierarchical and multi-track generation [52,57,64,70].”
Deep Learning Techniques for Music Generation - A Survey (2017, background): “The results show a very significant gain with the hierarchical architecture, see more details in [148] and [160].”