A Deep Residual Network for Large-Scale Acoustic Scene Analysis

By Logan Ford, Hao Tang, François Grondin, James R. Glass. 2019

52 citations recorded by the source corpus.

Many of the recent advances in audio event detection, particularly on the AudioSet data set, have focused on improving performance using the released embeddings produced by a pretrained model. In this work, we instead study the task of training a multi-label event classifier directly from the audio recordings of AudioSet. Using the audio recordings, not only are we able to reproduce results from prior work, we have also confirmed improvements of other proposed additions, such as an attention module. Moreover, by training the embedding network jointly with the additions, we achieve an mAP of 0.392 and an AUC of 0.971, surpassing the state of the art without transfer learning from a large data set. We also analyze the output activations of the network and find that the models are able to localize audio events when a finer time resolution is needed.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Perceiver: General Perception with Iterative Attention (2021, methodology): “Our latent Transformer uses the GPT-2 architecture (Rad-ford et al., 2019), which itself is based on the decoder of the original Transformer architecture (Vaswani et al., 2017).”
FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, neutral): “These results contrast with the successful results of [111, 123] for AudioSet classification.”
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019, methodology): “The recent DeepRes system [48] was built on waveforms downloaded from YouTube, and achieved an mAP of 0.”
Audio Retrieval with Natural Language Queries (2021, background): “Of relevance to our approach, a number of prior works have employed deep learning for audio comprehension [18, 19, 20, 21, 22] on the AudioSet dataset.”
The Benefit of Temporally-Strong Labels in Audio Event Classification (2021, methodology): “Other works use strategies including pooling [9] and attention [12] to train on weak labels while still producing scores at fine time resolutions.”
Multi-Format Contrastive Learning of Audio Representations (2021, background): “Such spectrograms can further be retrieved into log-mel filter banks and mel-frequency cepstral coefficients (MFCCs), which are the dominant formats for supervised and unsupervised audio recognition [16, 17, 18, 19, 20, 26].”
Large Scale Audiovisual Learning of Sounds with Weakly Labeled Data (2020, methodology): “Most of the recent methods rely on deep neural networks, in particular, deep Convolutional Neural Networks (ConvNets) [Kumar and Ithapu, 2020; Adavanne et al., 2019; Kong et al., 2019; Ford et al., 2019].”
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection (2022, unknown): “Convolutional neural networks (CNNs) have been widely used in this ﬁeld, such as DeepResNet [9], TALNet [10], PANN [11], and PSLA [12].”
Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking (2020, background): “…noise problems, they have been directly addressed in only a few of the previous works using AudioSet (e.g., [13, 14]), while the majority of efforts focus on deriving more sophisticated network architectures that ignore or downplay the idiosyncrasies of the labeled audio data (e.g., [15, 16]).”