Pop Music Transformer: Generating Music with Rhythm and Harmony

By Yu-Siang Huang, Yi-Hsuan Yang. 2020

40 citations recorded by the source corpus.

The task automatic music composition entails generative modeling of music in symbolic formats such as the musical scores. By serializing a score as a sequence of MIDI-like events, recent work has demonstrated that state-of-the-art sequence models with self-attention work nicely for this task, especially for composing music with long-range coherence. In this paper, we show that sequence models can do even better when we improve the way a musical score is converted into events. The new event set, dubbed "REMI" (REvamped MIDI-derived events), provides sequence models a metric context for modeling the rhythmic patterns of music, while allowing for local tempo changes. Moreover, it explicitly sets up a harmonic structure and makes chord progression controllable. It also facilitates coordinating different tracks of a musical piece, such as the piano, bass and drums. With this new approach, we build a Pop Music Transformer that composes Pop piano music with a more plausible rhythmic structure than prior arts do. The code, data and pre-trained model are publicly available.\footnote{\url{this https URL}}

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Monica Dinculescu
2. Carrie J. Cai
3. Cheng-Zhi Anna Huang

In Computer Vision and Pattern Recognition

1. Monica Dinculescu
2. Carrie J. Cai
3. Ed Newton-Rex
4. Yixiao Zhang
5. Yiyi Zhang
6. Ruihan Yang
7. Ziyu Wang
8. Gus G. Xia

In Signal Processing

1. Monica Dinculescu
2. Ed Newton-Rex
3. Carrie J. Cai
4. Yixiao Zhang
5. Yiyi Zhang
6. Ruihan Yang
7. Ziyu Wang
8. Gus G. Xia

Citations in the corpus

• Human-AI Co-Creation in Songwriting (2020, background): “More recently, a proliferation of modern deep learning techniques [8,9] has enabled models capable of generating full scores [40], or producing music that is coherent to both local and distant regions of music [39, 55].”
• Relative Positional Encoding for Transformers with Linear Complexity (2021, unknown): “Several variants for RPE were introduced (Huang et al., 2020; Wang et al., 2021).”
• PIANOTREE VAE: Structured Representation Learning for Polyphonic Music (2020, methodology): “Models MIDI-like event sequence is first used in deep music generation in performanceRNN [24] and Multi-track Music-VAE [9], and then broadly applied in transformer-based generation [19, 25, 26].”