Unsupervised Contrastive Learning of Sound Event Representations

By Eduardo Fonseca, Diego Ortego, Kevin McGuinness, Noel Edward O'Connor, Xavier Serra. 2020

71 citations recorded by the source corpus.

Self-supervised representation learning can mitigate the limitations in recognition tasks with few manually labeled data but abundant unlabeled data—a common scenario in sound event research. In this work, we explore unsupervised contrastive learning as a way to learn sound event representations. To this end, we propose to use the pretext task of contrasting differently augmented views of sound events. The views are computed primarily via mixing of training examples with unrelated backgrounds, followed by other data augmentations. We analyze the main components of our method via ablation experiments. We evaluate the learned representations using linear evaluation, and in two in-domain downstream sound event classification tasks, namely, using limited manually labeled data, and using noisy labeled data. Our results suggest that unsupervised contrastive pre-training can mitigate the impact of data scarcity and increase robustness against noisy labels.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, background): “made in the development of sound event recognizers with noisy supervision [32, 42] or self-supervision [49]–[51].”
Multimodal Self-Supervised Learning of General Audio Representations (2021, unknown): “In [14, 15, 16] however, the contrastive framework has been shown to be very effective on learning audio representations.”
BEATs: Audio Pre-Training with Acoustic Tokenizers (2022, methodology): “The self-supervised audio pre-training methods typically learn the audio representations with the contrastive learning (Ravanelli & Bengio, 2018; Saeed et al., 2021; Fonseca et al., 2021; Al-Tahan & Mohsenzadeh, 2021; Wang & Oord, 2021) or reconstruction objective (Tagliasacchi et al.”
Pengi: An Audio Language Model for Audio Tasks (2023, methodology): “To leverage from larger amounts of audio that is unlabeled, the community has employed Self-Supervised and Unsupervised Learning [52, 53, 46, 21].”
Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks (2021, methodology): “We choose mixup because the concept of mixing sounds is an audio-informed operation, and it has been proven useful for SET [2, 28, 3] and other sound event research tasks [29].”
Self-Supervised Learning from Automatically Separated Sound Scenes (2021, methodology): “We draw negative examples from clips within the batch [16, 13, 10, 14].”