Multi-level attention model for weakly supervised audio classification

By Changsong Yu, Karim Said Barsim, Qiuqiang Kong, Binh Yang. 2018

84 citations recorded by the source corpus.

In this paper, we propose a multi-level attention model for the weakly labelled audio classification problem. The objective of audio classification is to predict the presence or the absence of sound events in an audio clip. Recently, Google published a large scale weakly labelled AudioSet dataset containing 2 million audio clips with only the presence or the absence labels of the sound events, without the onset and offset time of the sound events. Previously proposed attention models only applied a single attention module on the last layer of a neural network which limited the capacity of the attention model. In this paper, we propose a multi-level attention model which consists of multiple attention modules applied on the intermediate neural network layers. The outputs of these attention modules are concatenated to a vector followed by a fully connected layer to obtain the final prediction of each class. Experiments show that the proposed multi-attention attention model achieves a state-of-the-art mean average precision (mAP) of 0.360, outperforming the single attention model and the Google baseline system of 0.327 and 0.314, respectively.

Subfields: Computer Vision and Pattern Recognition, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Speech & Audio

In Signal Processing

Citations in the corpus

Perceiver: General Perception with Iterative Attention (2021, neutral): “Audio Video A+V Benchmark (Gemmeke et al., 2017) 31.4 - - Attention (Kong et al., 2018) 32.7 - - Multi-level Attention (Yu et al., 2018) 36.0 - - ResNet-50 (Ford et al., 2019) 38.0 - - CNN-14 (Kong et al., 2020) 43.1 - - CNN-14 (no balancing & no mixup) (Kong et al., 2020) softmax outputs are very s”
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019, background): “Several works [15], [16], [17], [18], [19] have investigated building systems using the embedding features [14].”
SeCoST: Sequential Co-Supervision for Weakly Labeled Audio Event Detection (2019, background): “Some are driven by attention mechanisms in neural networks [11], so as to efﬁciently characterize the temporal occurrences of events in the audio recordings [8], [12], [13].”
Behavioral Use Licensing for Responsible AI (2020, methodology): “It has been trained using theAudioSet Dataset [30] and uses amulti-level attention based classifier [72].”
A General Survey on Attention Mechanisms in Deep Learning (2022, background): “Another domain where attention can be applied is audio processing [24], [25].”
A Deep Residual Network for Large-Scale Acoustic Scene Analysis (2019, methodology): “This work is extended by [22], applying a multi-level attention module.”
What Makes Training Multi-Modal Classification Networks Hard? (2019, methodology): “G-Blend outperforms two state-of-the-art Multi-level Attention Network[56] and TAL-Net[53] by 5.8% and 5.5 % on mAP respectively, although the ﬁrst one uses strong features (pre-trained on YouTube100M) and the second uses 100 clips per video, while G-Blend uses only 10.”
Audio Retrieval with Natural Language Queries (2021, background): “Of relevance to our approach, a number of prior works have employed deep learning for audio comprehension [18, 19, 20, 21, 22] on the AudioSet dataset.”