Audio Set: An ontology and human-labeled dataset for audio events

By Jort Florent Gemmeke, D. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, Marvin Ritter. 2017

4,118 citations recorded by the source corpus.

Subfields: Music, Evaluation & Benchmarks, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Music

1. Xavier Serra
2. Eduardo Fonseca
3. Xavier Favory
4. Frederic Font
5. Andrew Zisserman
6. Anurag Kumar
7. Manoj Plakal
8. Justin Salamon

In Evaluation & Benchmarks

1. Kristen Grauman
2. Andrew Zisserman
3. Antonio Torralba
4. Xavier Serra
5. Lorenzo Torresani
6. Eduardo Fonseca
7. Kaiming He
8. Yejin Choi

In Signal Processing

1. Jinyu Li
2. James R. Glass
3. Shinji Watanabe
4. Dong Yu
5. Wei-Ning Hsu
6. Yu Zhang
7. Tara N. Sainath
8. Yossi Adi

Citations in the corpus

• Self-Supervised MultiModal Versatile Networks (2020, methodology): “To evaluate the quality of the audio representation, we use the ESC-50 [66] (2K audio clips with 50 classes) and AudioSet [25] (20K eval audio clips with 527 classes) classification task using the frozen setting on the features produced by the last convolution of the audio backbone network.”
• Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, background): “, 2020) and joint audio-video classification on AudioSet (Gemmeke et al., 2017).”
• Perceiver: General Perception with Iterative Attention (2021, methodology): “We experimented with audio event classification in video using the AudioSet dataset (Gemmeke et al., 2017) – a large dataset with 10s long 1.”
• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “For the AudioSet benchmark [37], which involves a wide variety of non-speech sounds, we find intermediate Conformer layers pre-trained on the non-labeled native dataset, rather than YT-U, to yield SoTA results.”
• Learning Robust and Multilingual Speech Representations (2020, neutral): “Unlabeled speech pretraining corpus For pretraining, we collected a diverse and noisy speech corpus from several existing datasets: the subset of Audio Set (Gemmeke et al., 2017) containing speech examples, the audio part of AVSpeech (Ephrat et al.”
• MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, methodology): “For longer audio clips, we encode multiple CLS representations and then average the supervised data from AudioSet [42].”
• Florence: A New Foundation Model for Computer Vision (2021, neutral): “, 2019), AudioSet (Gemmeke et al., 2017), ImageNet (Deng et al.”
• Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning (2017, positive): “13 We quantify the number of object-selective units for our Cluster method trained on AudioSet (Gemmeke et al. 2017).”
• Continual Lifelong Learning with Neural Networks: A Review (2018, methodology): “3b) and the AudioSet dataset, which is built from YouTube videos with 10-s sound clips from 632 classes and over 2million annotations (Gemmeke et al., 2017).”
• High Fidelity Neural Audio Compression (2022, methodology): “For general audio, we use on AudioSet (Gemmeke et al., 2017) together with FSD50K (Fonseca et al., 2021).”
• MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (2017, methodology): “Finally, as a dataset encompassing a different domain, we evaluate on AudioSet [7].”
• FearNet: Brain-Inspired Model for Incremental Learning (2017, neutral): “AudioSet is an audio classification dataset (Gemmeke et al., 2017).”
• Measuring Catastrophic Forgetting in Neural Networks (2017, background): “AudioSet AudioSet (Gemmeke et al. 2017) is a hierarchically organized audio classification dataset built from YouTube videos.”
• Transformers in Vision: A Survey (2021, methodology): “The proposed model is pre-trained on Kinetics-700 [150] dataset and later finetuned on downstream video classification tasks such as short video classification on UCF101 [157], audio classification on ESC50 [158] and long-term action recognition on Charades [159] and Kinetics-Sounds [57] datasets.”
• Continual Lifelong Learning in Natural Language Processing: A Survey (2020, methodology): “Benchmark corpora have also been proposed for objects - CORE50 (Lomonaco and Maltoni, 2017) and sound - AUDIOSET (Gemmeke et al., 2017).”
• Audiobox: Unified Audio Generation with Natural Language Prompts (2023, background): “First, we annotate a set of 1,946 recordings sampled from diverse sources, including LibriTTS (Zen et al., 2019), Common Voice (Ardila et al., 2019), Switchboard (Godfrey et al., 1992), Fisher (Cieri, Christopher, et al. , 2004,2005a,,), Spotify (Clifton et al., 2020), AudioSet (Gemmeke et al., 2017”