Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints

By Stefan Lattner, Maarten Grachten, Gerhard Widmer. 2016

73 citations recorded by the source corpus.

We introduce a method for imposing higher-level structure on generated, polyphonic music. A Convolutional Restricted Boltzmann Machine (C-RBM) as a generative model is combined with gradient descent constraint optimisation to provide further control over the generation process. Among other things, this allows for the use of a "template" piece, from which some structural properties can be extracted, and transferred as constraints to the newly generated material. The sampling process is guided with Simulated Annealing to avoid local optima, and to find solutions that both satisfy the constraints, and are relatively stable with respect to the C-RBM. Results show that with this approach it is possible to control the higher-level self-similarity structure, the meter, and the tonal properties of the resulting musical piece, while preserving its local musical coherence.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

Music Transformer: Generating Music with Long-Term Structure (2018, background): “Lattner et al. (2018) use self-similarity in style-transfer fashion, where the self-similarity structure of a piece serves as a template objective for gradient descent to impose similar repetition structure on an input score.”
An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation (2018, methodology): “Selfsimilarity has been used for example in [16] in a style-transfer like fashion where the self-similarity structure of a piece serves as a template objective for gradient descent to modify an input score to bear similar repetition structure.”
Modeling Self-Repetition in Music Generation using Generative Adversarial Networks (2019, background): “Widmer et al (2018) use a convolutional restricted boltzmann machine to generate music while imposing a given repetition structure of a piece.”
Deep Learning Techniques for Music Generation - A Survey (2017, methodology): “in [100], the starting point is to use a restricted Boltzmann machine (RBM) to learn the local structure, seen as the musical texture, of a corpus of musical pieces.”
Counterpoint by Convolution (2019, background): “Lattner et al. imposes higher-level structure by interleaving selective Gibbs sampling on a convolutional RBM [26] and gradient descent that minimizes cost to template piece on features such as self-similarity.”