Audio event and scene recognition: A unified approach using strongly and weakly labeled data

By Bhiksha Raj, Anurag Kumar. 2016

41 citations recorded by the source corpus.

In this paper we propose a novel learning framework called Supervised and Weakly Supervised Learning where the goal is to learn simultaneously from weakly and strongly labeled data. Strongly labeled data can be simply understood as fully supervised data where all labeled instances are available. In weakly supervised learning only data is weakly labeled which prevents one from directly applying supervised learning methods. Our proposed framework is motivated by the fact that a small amount of strongly labeled data can give considerable improvement over only weakly supervised learning. The primary problem domain focus of this paper is acoustic event and scene detection in audio recordings. We first propose a naive formulation for leveraging labeled data in both forms. We then propose a more general framework for Supervised and Weakly Supervised Learning (SWSL). Based on this general framework, we propose a graph based approach for SWSL. Our main method is based on manifold regularization on graphs in which we show that the unified learning can be formulated as a constraint optimization problem which can be solved by iterative concave-convex procedure (CCCP). Our experiments show that our proposed framework can address several concerns of audio content analysis using weakly labeled data.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, methodology): “…approaches using audio and text information (e.g., using the provided Freesound tags, title, and textual description for the clips) [91, 92]; evaluation of hierarchical classiﬁcation via ontology-aware learning frameworks [5, 93, 94]; or approaches speciﬁcally combining strong and weak labels [95].”
The Benefit of Temporally-Strong Labels in Audio Event Classification (2021, background): “A performance boost is reported by adding a small amount of strongly labeled data to the weak labels, possibly due to the mitigation of the inherent problems in weakly labeled data [13, 14, 15].”
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer (2021, background): “Others use weaker forms of supervision for labeling (Kumar and Raj, 2017; Kong et al., 2018) and localizing (McFee et al.”
Learning Sound Event Classifiers from Web Audio with Noisy Labels (2019, background): “In [11], classifiers are learnt from weakly labeled web data, and to improve performance an approach is proposed using a small amount of strongly labeled audio along with the web data.”
Sound Event Detection Using Point-Labeled Data (2019, background): “As a result, models that can be trained on weakly labeled data have obtained much attention for sound event classification and detection [8, 4, 9, 10, 11].”
Freesound Datasets: A Platform for the Creation of Open Audio Datasets (2017, background): “16 Based on these two conditions, we can assume that most of those audio clips are just examples for those acoustic events, and can be considered strongly labeled data [15].”
A Closer Look at Weak Label Learning for Audio Events (2018, background): “The constraint optimization problem described in [43] work can be very difficult to solve on such”