FSD50K: An Open Dataset of Human-Labeled Sound Events
By Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, Xavier Serra. 2020
737 citations recorded by the source corpus.
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2 M tracks from YouTube videos and encompassing over 500 sound classes. However, AudioSet is not an open dataset as its official release consists of pre-computed audio features. Downloading the original audio tracks can be problematic due to YouTube videos gradually disappearing and usage rights issues. To provide an alternative benchmark dataset and thus foster SER research, we introduce FSD50K, an open dataset containing over 51 k audio clips totalling over 100 h of audio manually labeled using 200 classes drawn from the AudioSet Ontology. The audio clips are licensed under Creative Commons licenses, making the dataset freely distributable (including waveforms). We provide a detailed description of the FSD50K creation process, tailored to the particularities of Freesound data, including challenges encountered and solutions adopted. We include a comprehensive dataset characterization along with discussion of limitations and key factors to allow its audio-informed usage. Finally, we conduct sound event classification experiments to provide baseline systems as well as insight on the main factors to consider when splitting Freesound audio data for SER. Our goal is to develop a dataset to be widely adopted by the community as a new open benchmark for SER research.
Subfields: Music, Evaluation & Benchmarks, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Music
- Eduardo Fonseca
- Xavier Serra
- Jakob Abeßer
- Benjamin Elizalde
- Huaming Wang
- Justin Salamon
- Qiuqiang Kong
- Mark D. Plumbley
In Evaluation & Benchmarks
- Xavier Serra
- Eduardo Fonseca
- Jakob Abeßer
- Mark D. Plumbley
- Benjamin Elizalde
- Yossi Adi
- Huaming Wang
- D. Ellis
In Speech & Audio
- Yossi Adi
- Gabriel Synnaeve
- Jade Copet
- Xavier Serra
- Alexandre D'efossez
- Adam Polyak
- Eduardo Fonseca
- Felix Kreuk
In Signal Processing
- Eduardo Fonseca
- Xavier Serra
- Yossi Adi
- John R. Hershey
- Scott Wisdom
- Adam Polyak
- Gabriel Synnaeve
- Hakan Erdogan
Citations in the corpus
- High Fidelity Neural Audio Compression (2022, methodology): “For general audio, we use on AudioSet (Gemmeke et al., 2017) together with FSD50K (Fonseca et al., 2021).”
- AudioGen: Textually Guided Audio Generation (2022, background): “We use a set of several datasets: AudioSet (Gemmeke et al., 2017), BBC sound effects 1 , AudioCaps (Kim et al., 2019), Clotho v2 (Drossos et al., 2020), VGG-Sound (Chen et al., 2020), FSD50K (Fonseca et al., 2021), Free To Use Sounds 2 , Sonniss Game Effects 3 , WeSoundEffects 4 , Paramount Motion -”
- Separate Anything You Describe (2023, neutral): “Each mixture is assigned one of the five audio captions of the target source.”
- USM-SED - A Dataset for Polyphonic Sound Event Detection in Urban Sound Monitoring Scenarios (2021, methodology): “These soundscapes were created by systematically mixing isolated sound recordings taken from the public FSD50K dataset [5].”
- CLAP Learning Audio Concepts from Natural Language Supervision (2023, methodology): “We extracted 36,796 pairs from FSD50k [17] (training and validation splits; title and description were concatenated); 29,646 pairs from ClothoV2 [18] (each audio has 5 captions so we created 5 pairs per clip); 44,292 from AudioCaps [19]; 17,276 pairs from MACS [20] (same process to create pairs as…”
- What’s all the Fuss about Free Universal Sound Separation Data? (2020, methodology): “The audio data is sourced from a subset of FSD50K [19], a sound event dataset composed of freesound.org content annotated with labels from the AudioSet ontology [20].”
- WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research (2023, background): “O VER the past decade, the field of machine listening has achieved notable progress, with the aid of deep learning techniques and the availability of audio event datasets [1], [2], [3].”
- LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT (2023, methodology): “…utterances are extracted from the AISHELL-1, AISHELL-2, LibriSpeech, and WSJ datasets Paul & Baker (1992), while the noisy counterparts are generated by mixing the clean speech with noises from the FSD-50K dataset (Fonseca et al., 2022) at random signal-to-noise rates (SNR) ranging from 2 to 15.”
- Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks (2021, methodology): “We evaluate the proposed methods on a large-vocabulary sound event tagging task using the recently released FSD50K dataset [16].”
- Improving Sound Event Detection in Domestic Environments using Sound Separation (2020, methodology): “The foreground events are obtained from FSD50k [19, 20].”
- Wav2CLIP: Learning Robust Audio Representations from Clip (2021, neutral): “FSD50K [25] is 5x larger than UrbanSound8K, and is a more challenging dataset with multi-labels.”
- Efficient Training of Audio Transformers with Patchout (2021, methodology): “(Tagging) in FSD50K [28](mean average precision).”
- Pengi: An Audio Language Model for Audio Tasks (2023, background): “[63], MELD [48]; Music Analysis: NSynth [19], FMA [9]; Audio Captioning: AudioCaps [32], ClothoV2 [15]; Audio Question and Answering: ClothoAQA [39]; Auxiliary: WavText5K [12], SoundDescs [35], MACS [42], WavCaps [43], FreeSound [20] and FindSound 2 .”
- Unsupervised Contrastive Learning of Sound Event Representations (2020, methodology): “Clips coming from the same user are likely to share some of these patterns, which has been shown to have an impact in supervised sound event tagging [2].”
- AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models (2023, neutral): “The datasets used in our work consist of AudioCaps [22], AudioSet [12], FSD50K [11], and ESC50 [41].”