Detection and Classification of Acoustic Scenes and Events

By Dan Stowell, Dimitrios Giannoulis, Emmanouil Benetos, Mathieu Lagrange, Mark D. Plumbley. 2015

748 citations recorded by the source corpus.

For intelligent systems to make best use of the audio modality, it is important that they can recognize not just speech and music, which have been researched as specific tasks, but also general sounds in everyday environments. To stimulate research in this field we conducted a public research challenge: the IEEE Audio and Acoustic Signal Processing Technical Committee challenge on Detection and Classification of Acoustic Scenes and Events (DCASE). In this paper, we report on the state of the art in automatically classifying audio scenes, and automatically detecting and classifying audio events. We survey prior work as well as the state of the art represented by the submissions to the challenge from various research groups. We also provide detail on the organization of the challenge, so that our experience as challenge hosts may be useful to those organizing challenges in similar domains. We created new audio datasets and baseline systems for the challenge; these, as well as some submitted systems, are publicly available under open licenses, to serve as benchmarks for further research in general-purpose machine listening.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Moments in Time Dataset: One Million Videos for Event Understanding (2018, background): “[41] collected an early dataset and assembled a challenge for sound classification, Piczak [32] collected a dataset of fifty sound categories and enough to train deep convolutional models, Salamon et al.”
Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (2018, methodology): “For this purpose, we take the activations from the conv_5 layer of the audio subnetwork and test their quality as audio representation on two established sound classiﬁcation datasets: ESC-50 [29] and DCASE2014 [30].”
Audio-Visual Instance Discrimination with Cross-Modal Agreement (2020, methodology): “Audio representations are evaluated on the ESC-50 [62] and DCASE [74] datasets by linear probing [22] for the task of sound recognition.”
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019, background): “Recently, the Detection and Classification of Acoustic Scenes and Events (DCASE) challenge series [7][8][9][2] have provided publicly available datasets, such as acoustic scene classification and sound event detection datasets.”
Self-supervised audio representation learning for mobile devices (2019, methodology): “Finally, we use two datasets released in the context of the recent DCASE2018 Challenge, Bird Audio Detection [39] and TUT Urban Acoustic Scenes 2018 [25], which contains labeled audio samples from 10 different urban environments.”
Automatic acoustic detection of birds through deep learning: The first Bird Audio Detection challenge (2018, methodology): “It has been superseded for accuracy in general-purpose sound recognition bymore advancedmethods (Stowell et al., 2015).”
Self-Supervised Speech Representation Learning: A Review (2022, methodology): “2018 [249]) plus an SID task built with the LS train-clean-100 subset to evaluate the capability of representations on audio event detection.”
Robust sound event detection in bioacoustic sensor networks (2019, background): “As a result, the application domain of digital audio technologies has extended far beyond the scope of interhuman communication to encompass the development of new cyberphysical systems [5].”
Sound event detection in synthetic audio: Analysis of the dcase 2016 task results (2017, methodology): “01), but no significant effect of polyphony (F [1, 52] = 3.”
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey (2019, methodology): “Variational TABLE 1 Summary of Commonly Used Datasets of Images, Videos, Audios, and 3D Object Data Dataset Data Type Size Synthetic # classes Groundtruth Labeling CIFAR10 [74] Image 60,000 Images No 10 Object category label ImageNet [14] Image 1.3 million images No 1,000 Object category label MNIST”
Audio event and scene recognition: A unified approach using strongly and weakly labeled data (2016, background): “Several acoustic feature representations for audio event or scene detection has been proposed [1].”
Multi-level attention model for weakly supervised audio classification (2018, background): “The detection and classification of acoustic scenes and events (DCASE) challenge 2013, 2016, 2017 [1, 2, 6] datasets consists of several hours of data.”
A Framework for the Robust Evaluation of Sound Event Detection (2019, methodology): “The notion of collars introduced in [9,10] and deployed in DCASE [4, 5, 7, 8] considers that a ground truth event for class c, yi ∈ Yc is correctly detected as a TP if the following condition is satisfied: ∃xj ∈ X ∗ c such that (ts,i − tc) ≤ ts,j ≤ (ts,i + tc) AND (te,i − t̄c) ≤ te,j ≤ (te,i + t̄c) ”
Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016 Challenge (2018, background): “For DCASE 2013, the organizers indeed reported various software issues with libraries, Linux/Windows differences, formatting and bugs in the submitted code, which are all avoided by requiring submission of system output [38].”
The SINS Database for Detection of Daily Activities in a Home Environment Using an Acoustic Sensor Network (2017, methodology): “The LITIS Rouen Audio Scene dataset [12], DCASE 2013 and 2016 databases [13, 14] consist of (binaural) recordings of events and/or scenes in public areas, e.”
Look, Listen, and Learn More: Design Choices for Deep Audio Embeddings (2019, background): “As a result, most existing machine listening datasets are relatively small and narrowly focused [1, 13, 18], and only recently has the community seen a large-scale dataset with the introduction of AudioSet [7].”
SeCoST: Sequential Co-Supervision for Weakly Labeled Audio Event Detection (2019, background): “While several classical machine learning and deep learning methods have been developed for AED using strong labels [2], [1], [3], [4], much of the recent progress has focused on efficiently leveraging weakly labeled data [5].”