Piano Genie

By Chris Donahue, Ian Simon, Sander Dieleman. 2018

51 citations recorded by the source corpus.

We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons, and their performance is decoded into the space of plausible piano music in real time. To learn a suitable mapping procedure for this problem, we train recurrent neural network autoencoders with discrete bottlenecks: an encoder learns an appropriate sequence of buttons corresponding to a piano piece, and a decoder learns to map this sequence back to the original piece. During performance, we substitute a user's input for the encoder output, and play the decoder's prediction each time the user presses a button. To improve the intuitiveness of Piano Genie's performance behavior, we impose musically meaningful constraints over the encoder's outputs.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Continuous diffusion for categorical data (2022, methodology): “We experimented with L2 regularisation, margin penalties (Dieleman et al., 2021; Donahue et al., 2019), as well as automatic adaptation of the L2 penalty weight to target unit variance for the embedding vector components (Dieleman et al., 2021; Rezende and Viola, 2018), but found that normalisation…”
Deep Learning for Audio Signal Processing (2019, methodology): “For controlled audio synthesis [64], one loss function was customized to encourage the latent variables of a variational autoencoder (VAE) to remain inside a defined range and another to have changes in the control space be reflected in the generated audio.”
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models (2020, background): “…a drawing that a user has started [36, 9, 32, 14] or ﬁlling in a missing section of a song [27, 24], generative models could enable untrained lay users to take part in creative experiences that would otherwise be difﬁcult to achieve without additional training or specialization [29, 12, 19].”
Variable-rate discrete representation learning (2021, background): “Following Donahue et al. (2019), we add a loss term L margin ( z ) = max ( | z | − 1 , 0) , which is non-zero only when z is not in the valid range.”
Human-AI Co-Creation in Songwriting (2020, background): “exploring more adventurous chord progressions [27, 37, 49], semantic controls to music and sound generation [11, 24, 36, 63], building new instruments through custom mappings or unsupervised learning [20,25], and enabling experimentation in all facets of symbolic and acoustic manipulation of the mus”