Music Transformer 기반 음악

By S. Cho, Jaeho Lee

Subfields: Generative Models, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Speech & Audio

Citations in the corpus

RoFormer: Enhanced Transformer with Rotary Position Embedding (2021, neutral): “On the other side, the previous work Parikh et al. [2016], Shaw et al. [2018], Huang et al. [2018], Dai et al. [2019], Yang et al. [2019], Raffel et al. [2020], Ke et al. [2020], He et al. [2020], Huang et al. [2020] focuses on relative position encoding, which typically encodes the relative…”
Stand-Alone Self-Attention in Vision Models (2019, neutral): “We achieve this in two steps:”
Pix2seq: A Language Modeling Framework for Object Detection (2021, background): “…of modeling various sequential data, such as machine translation (Sutskever et al., 2014; Bahdanau et al., 2014), image captioning (Vinyals et al., 2015b; Karpathy & Fei-Fei, 2015; Xu et al., 2015), and many others (Vinyals et al., 2015a; Huang et al., 2018; Ramesh et al., 2021; Chen et al., 2021).”
Synthesizer: Rethinking Self-Attention for Transformer Models (2020, background): “For example, recent studies have investigated improving the layer’s overall efﬁciency via sparsiﬁcation and reducing the complexity of computing the alignment matrix (Child et al., 2019; Kitaev et al., 2020; Huang et al., 2018; Tay et al., 2020; Beltagy et al., 2020).”
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers (2019, methodology): “Following (Huang et al., 2019; Zhang et al., 2019), we feed X to the BERT and use the last hidden states as the initial representations before proceeding with the RAT layers.”
GMAT: Global Memory Augmentation for Transformers (2020, methodology): “Moreover, Transformers with tailored attention patterns have been successfully used to replace convolutions in computer vision [16, 15], and have also been useful in music generation [7], symbolic mathematics [11] and other modalities.”
Variable-rate discrete representation learning (2021, background): “They have signiﬁcantly advanced the state of the art since (Simon & Oore, 2017; Huang et al., 2019; Payne, 2019; Oore et al., 2020).”
Neural Shufﬂe-Exchange Networks − Sequence Processing in O( n log n ) Time (2019, background): “A similar phenomenon occurs in music, where a common motif may reappear throughout the entire piece and it should be kept coherent by any applied transformation (Huang et al., 2019).”
Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy (2021, background): “Recently, Transformers (Vaswani et al., 2017) have shown great power in sequential data processing, such as natural language processing (Devlin et al., 2019; Brown et al., 2020), audio processing (Huang et al., 2019) and computer vision (Dosovitskiy et al., 2021; Liu et al., 2021).”
The Lipschitz Constant of Self-Attention (2020, background): “…various data modalities, starting with natural-language processing (Vaswani et al., 2017; Devlin et al., 2019; Brown et al., 2020) and extending to computer vision (Zhang et al., 2019; Parmar et al., 2019), audio generation (Huang et al., 2019), and reinforcement learning (Parisotto et al., 2020).”
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021, background): “Recently, Transformers [41, 45] based on the self-attention mechanism shows great power in sequential data, such as natural language processing [13, 8], audio processing [19] and even computer vision [16, 27].”
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models (2021, background): “They reached improved BLEU scores in machine translation, but their approach (and reﬁnements by Huang et al. (2019)) are hard to parallelize, which is unattractive in a world driven by parallel computing.”
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models (2020, background): “Research has demonstrated the potential for modeling and synthesizing music, ranging from single-voice sequences [13] and multi-part music [19, 34], to music with variable parts at each time step [4] and music with long-term structure over minutes [30, 37, 26].”
A Survey of Transformers (2021, methodology): “As an incremental effort, Music Transformer [56] further introduce a mechanism to reduce the intermediate memory requirements for this approach.”
Relative Positional Encoding for Transformers with Linear Complexity (2021, unknown): “Multiplicative strategies were proposed for both RPE (Huang et al., 2020) and APE (Dai et al., 2019).”
Human-AI Co-Creation in Songwriting (2020, background): “More recently, a proliferation of modern deep learning techniques [8,9] has enabled models capable of generating full scores [40], or producing music that is coherent to both local and distant regions of music [39, 55].”
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, background): “More generally, transformers have proven capable in various NLP and speech tasks (Dong et al., 2018; Huang et al., 2019; Le et al., 2019; Li et al., 2019).”