Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms

By Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim, Juhan Nam. 2017

206 citations recorded by the source corpus.

Recently, the end-to-end approach that learns hierarchical representations from raw data using deep convolutional neural networks has been successfully explored in the image, text and speech domains. This approach was applied to musical signals as well but has been not fully explored yet. To this end, we propose sample-level deep convolutional neural networks which learn representations from very small grains of waveforms (e.g. 2 or 3 samples) beyond typical frame-level input representations. Our experiments show how deep architectures with sample-level filters improve the accuracy in music auto-tagging and they provide results comparable to previous state-of-the-art performances for the Magnatagatune dataset and Million Song Dataset. In addition, we visualize filters learned in a sample-level DCNN in each layer to identify hierarchically learned features and show that they are sensitive to log-scaled frequency along layer, such as mel-frequency spectrogram that is widely used in music classification systems.

Subfields: Artificial Intelligence, Music, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Music

In Signal Processing

Citations in the corpus

A Wavenet for Speech Denoising (2017, methodology): “To this end, discriminative models have been used in a end-to-end learning fashion for music [6, 15] or speech classification [5, 20, 39].”
Adversarial Audio Synthesis (2018, background): “While a minority of recent research in discriminative audio classiﬁcation tasks has used raw audio input (Sainath et al., 2015; Lee et al., 2017), most of these approaches operate on spectrogram representations of audio.”
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019, methodology): “The original LeeNet consists of 11 layers.”
Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016 Challenge (2018, methodology): “classification methods, in particular CNNs [12], [29], [44].”
A Survey of Sound Source Localization with Deep Learning Methods (2021, methodology): “, a CNN applied on the time-domain signal samples) [Lee et al., 2017].”
Advancing Biosensors with Machine Learning. (2020, methodology): “Although the CNN is originally developed for 2D image recognition, the 1D CNN 88 achieved great advances, which have been shown to perform well on data having spatial relations in a single dimension, such as stock price, 89 electroencephalograms (EEGs), 90 audio signals, 91 and spectra.”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “Unfortunately, we were not able to replicate the per-class AUC results reported in [34], i.”
Evaluation of CNN-based Automatic Music Tagging Models (2020, methodology): “Sample-level CNN [3] pursued an assumption-free end-to-end model by applying 1D convolution directly to a raw audio waveform, and the following research [4] improved the performance by adding squeeze-and-excitation blocks [5].”
RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification (2019, background): “Other domains such as spoofing detection and automatic music tagging are also adopting raw waveform inputs [4, 8].”
Synthesizing Audio with Generative Adversarial Networks (2018, background): “While a minority of recent research in discriminative audio classification tasks has used raw audio input (Sainath et al., 2015; Lee et al., 2017), most of these approaches operate on spectrogram representations of audio.”
Deep Learning for Audio Signal Processing (2019, methodology): “[111] train a CNN on raw samples, using only short filters (size 2 to 4) interleaved with max-pooling, matching the performance of log-mel spectrograms.”
Exploiting time-frequency patterns with LSTM-RNNs for low-bitrate audio restoration (2019, methodology): “Owing to these successes, CNNs have been increasingly employed for acoustic analysis as well, such as speech recognition [3], speaker analysis [36], music information retrieval [25], and denoising [44].”