Audio Set Classification with Attention Model: A Probabilistic Perspective

By Qiuqiang Kong, Yong Xu, Wenwu Wang, Mark D. Plumbley. 2017

107 citations recorded by the source corpus.

This paper investigates the Audio Set classification. Audio Set is a large scale weakly labelled dataset (WLD) of audio clips. In WLD only the presence of a label is known, without knowing the happening time of the labels. We propose an attention model to solve this WLD problem and explain the attention model from a novel probabilistic perspective. Each audio clip in Audio Set consists of a collection of features. We call each feature as an instance and the collection as a bag following the terminology in multiple instance learning. In the attention model, each instance in the bag has a trainable probability measure for each class. The classification of the bag is the expectation of the classification output of the instances in the bag with respect to the learned probability measure. Experiments show that the proposed attention model achieves a mAP of 0.327 on Audio Set, outperforming the Google's baseline of 0.314.

Subfields: Computer Vision and Pattern Recognition, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Speech & Audio

In Signal Processing

Citations in the corpus

Perceiver: General Perception with Iterative Attention (2021, background): “Nevertheless, self-attention has been rapidly percolating into perception, for example as pieces of otherwise convolutional models for images (Bello et al., 2019; Cordonnier et al., 2020; Srinivas et al., 2021) and videos (Wang et al., 2018; Girdhar et al., 2019).”
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019, background): “Several researchers have investigated building systems with those embedding features [13], [16]–[20].”
Audio Retrieval with Natural Language Queries (2021, background): “Of relevance to our approach, a number of prior works have employed deep learning for audio comprehension [18, 19, 20, 21, 22] on the AudioSet dataset.”
Multi-level attention model for weakly supervised audio classification (2018, methodology): “[10] proposed an single-level attention model for audio classification and outperforms both the MIL method [9] and the Google baseline deep neural network system [7] on Audio Set classification.”
Implicit Neural Representations with Periodic Activation Functions (2020, background): “Other work explores neural networks with periodic activations for simple classification tasks [22–24] and recurrent neural networks [25–29].”
A Deep Residual Network for Large-Scale Acoustic Scene Analysis (2019, background): “The study in [21] applies an attention module for each class over time after a few additional transformations on top of the released features.”
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer (2021, background): “Others use weaker forms of supervision for tagging (Kumar and Raj, 2017; Kong et al., 2018) and localization (McFee et al., 2018; Kim and Pardo, 2019).”
Sound Event Detection Using Point-Labeled Data (2019, background): “As a result, models that can be trained on weakly labeled data have obtained much attention for sound event classification and detection [8, 4, 9, 10, 11].”