Multi-Level and Multi-Scale Feature Aggregation Using Pretrained Convolutional Neural Networks for Music Auto-Tagging
By Jongpil Lee, Juhan Nam. 2017
149 citations recorded by the source corpus.
Music auto-tagging is often handled in a similar manner to image classification by regarding the two-dimensional audio spectrogram as image data. However, music auto-tagging is distinguished from image classification in that the tags are highly diverse and have different levels of abstraction. Considering this issue, we propose a convolutional neural networks (CNN)-based architecture that embraces multi-level and multi-scaled features. The architecture is trained in three steps. First, we conduct supervised feature learning to capture local audio features using a set of CNNs with different input sizes. Second, we extract audio features from each layer of the pretrained convolutional networks separately and aggregate them altogether giving a long audio clip. Finally, we put them into fully connected networks and make final predictions of the tags. Our experiments show that using the combination of multi-level and multi-scale features is highly effective in music auto-tagging and the proposed method outperforms the previous state-of-the-art methods on the MagnaTagATune dataset and the Million Song Dataset. We further show that the proposed architecture is useful in transfer learning.
Subfields: Artificial Intelligence, Information Systems, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jordi Pons
- Xavier Serra
- Kyung-ah Sohn
- Byungkon Kang
- Dario Rethage
- Namhyuk Ahn
- Yossi Adi
- Lauri Juvela
In Information Systems
In Signal Processing
- Jordi Pons
- Xavier Serra
- Dario Rethage
- Eduardo Fonseca
- Yossi Adi
- Alexandre D'efossez
- Frederic Font
- Lauri Juvela
Citations in the corpus
- A Wavenet for Speech Denoising (2017, background): “This allows the network to explicitly incorporate features extracted at several hierarchical levels into its final prediction [14].”
- Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network (2018, background): “Multi-level representation is used in many deep learning methods [21, 22] because of its successful performance with simple modifications.”