Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
By Justin Salamon, Juan Pablo Bello. 2016
1,465 citations recorded by the source corpus.
The ability of deep convolutional neural networks (CNNs) to learn discriminative spectro-temporal patterns makes them well suited to environmental sound classification. However, the relative scarcity of labeled data has impeded the exploitation of this family of high-capacity models. This study has two primary contributions: first, we propose a deep CNN architecture for environmental sound classification. Second, we propose the use of audio data augmentation for overcoming the problem of data scarcity and explore the influence of different augmentations on the performance of the proposed CNN architecture. Combined with data augmentation, the proposed model produces state-of-the-art results for environmental sound classification. We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation. Finally, we examine the influence of each augmentation on the model's classification accuracy for each class, and observe that the accuracy for each class is influenced differently by each augmentation, suggesting that the performance of the model could be improved further by applying class-conditional data augmentation.
Subfields: Developmental Biology, Pulmonary and Respiratory Medicine, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Developmental Biology
- Dan Stowell
- Xavier Serra
- Eduardo Fonseca
- Reinhold Häb-Umbach
- Janek Ebbers
- Chang-Dong Yoo
- Jan van Gemert
- Hyunsin Park
In Pulmonary and Respiratory Medicine
- Xavier Serra
- Eduardo Fonseca
- Reinhold Häb-Umbach
- Janek Ebbers
- Chang-Dong Yoo
- Jan van Gemert
- Hyunsin Park
- Osman Semih Kayhan
In Signal Processing
- Xavier Serra
- Eduardo Fonseca
- Frederic Font
- Xavier Favory
- Jordi Pons
- James R. Glass
- Dan Stowell
- Justin Salamon
Citations in the corpus
- Recent advances in convolutional neural networks (2015, methodology): “Popular augmentation methods include simple geometric transformations such as sampling [7, 107], mirroring [106, 108], rotating [109], shifting [110], and various photometric transformations [111, 112].”
- Reinforcement Learning in Healthcare: A Survey (2019, methodology): “The direct solution can be using data augmentation strategies such as deformations [345] or GANs [346] to increase samples and then employ conventional learning methods.”
- FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, background): “Mitigating this potential issue could be a research problem involving, for example, data augmentation [82] or domain adaptation [83] techniques.”
- Acoustic Scene Classification by Ensembling Gradient Boosting Machine and Convolutional Neural Networks (2017, methodology): “Most CNN architectures proposed in the literature use squared filters and only one filter shape in the first convolutional layer [11, 12, 14].”
- Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring (2018, methodology): “Some studies have partially addressed this issue by augmenting training data with background noise to simulate different distances and acoustic environments (Salamon & Bello, 2016).”
- A scoping review of transfer learning research on medical image analysis using ImageNet (2020, methodology): “Data augmentation is a method for artificial data generation for training by creating variations of the original dataset [29].”
- Audioclip: Extending Clip to Image, Text and Audio (2021, methodology): “To successfully solve this task, different approaches were proposed that included the use of one- [27,28] or two-dimensional Convolutional Neural Networks (CNN) operating on static [18,24,32,9,15,17,33,8,30] or trainable [23,10] time-frequency transformation of raw audio.”
- Augmenting Image Classifiers Using Data Augmentation Generative Adversarial Networks (2018, background): “Two such domains is sound [32] and text [31].”
- Small Sample Learning in Big Data Era (2018, methodology): “For example, for audio data, Salamon and Bello (2017) applied four different deformations, namely, time stretching, pitch shifting, dynamic range compression, and background noise to overcome the problem of data scarcity.”
- Convolutional Recurrent Neural Network and Data Augmentation for Audio Tagging with Noisy Labels and Minimal Supervision (2019, positive): “It has been shown to improve classifier performance on many tasks, including speech recognition [6] and audio classification [7, 8, 9].”
- SONYC (2019, background): “25,26 Our research shows that feature learning, using even simple dictionary-based methods (such as spherical k-means) makes for significant improvement in performance over the traditional approach of feature engineering.”
- Learning Sound Event Classifiers from Web Audio with Noisy Labels (2019, methodology): “The model used is a CNN (3 conv + 1 dense) following that of [23] with two main changes.”
- Induction Networks for Few-Shot Text Classification (2019, background): “Early studies (Salamon and Bello, 2017) applied data augmentation and regularization techniques to alleviate the overfitting problem caused by data sparseness, only to a limited extent.”
- Attention Bottlenecks for Multimodal Fusion (2021, methodology): “The difference in input representations between audio and vision is particularly stark – many state of the art audio classification methods rely on short term Fourier analysis to produce log-mel spectrograms, often using them as inputs to CNN architectures designed for images [26, 51].”
- SpeechMix - Augmenting Deep Sound Recognition Using Hidden Space Interpolations (2020, methodology): “Sala-mon and Bello [5] proposed the usage of additional training data created by time stretching, pitch shifting, dynamic range compression, and adding background noise chosen from an external dataset.”
- COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations (2020, background): “DeepConv (Salamon & Bello, 2017) is a deep neural network composed of convolutional and max-pooling layers.”
- A Deep Residual Network for Large-Scale Acoustic Scene Analysis (2019, methodology): “In this work, we focus on deep convolutional neural networks (CNNs) for audio classification due to the success in many prior studies [14, 15, 16, 17, 18, 19].”
- Scaper: A library for soundscape synthesis and augmentation (2017, methodology): “[1] C. Mydlarz, J. Salamon, and J. P. Bello, “The implementation of low-cost urban acoustic monitoring devices,” Applied Acoustics, vol.”