The Benefit of Temporally-Strong Labels in Audio Event Classification

By Shawn Hershey, D. Ellis, Eduardo Fonseca, Aren Jansen, Caroline Liu, Channing Moore, Manoj Plakal. 2021

159 citations recorded by the source corpus.

To reveal the importance of temporal precision in ground truth audio event labels, we collected precise (∼0.1 sec resolution) "strong" labels for a portion of the AudioSet dataset. We devised a temporally-strong evaluation set (including explicit negatives of varying difficulty) and a small strong-labeled training subset of 67k clips (compared to the original dataset’s 1.8M clips labeled at 10 sec resolution). We show that fine-tuning with a mix of weak- and strongly-labeled data can substantially improve classifier performance, even when evaluated using only the original weak labels. For a ResNet-50 architecture, d′ on the strong evaluation data including explicit negatives improves from 1.13 to 1.39. The new labels are available as an update to AudioSet.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, background): “10 Recently, strong labels for a small portion of AudioSet (≈81k clips) were released [40].”
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection (2022, background): “In the future, we notice that a partial strong labeled subset of AudioSet has just been released [27], we decide to conduct a detail localization training and evaluation work by HTS-AT to further explore its potential.”
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research (2023, methodology): “Incontrast toonline-harvesteddatasetssuchasSoundDescsandLAION-Audio-630k,ourmethodentailsconvertingrawdescriptionsinto captionsandremovingnoisyorsound-unrelateddescriptions.”
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark (2024, unknown): “Using 100 unique sounds from the AudioSet Strong evaluation set (Hershey et al., 2021), we overlayed each sound with every other sound, ensuring variation in attributes such as loudness to enhance distinctiveness.”