Scaper: A library for soundscape synthesis and augmentation

By Justin Salamon, Duncan MacConnell, Mark Brozier Cartwright, Peter Qi Li, Juan Pablo Bello. 2017

247 citations recorded by the source corpus.

Sound event detection (SED) in environmental recordings is a key topic of research in machine listening, with applications in noise monitoring for smart cities, self-driving cars, surveillance, bioa-coustic monitoring, and indexing of large multimedia collections. Developing new solutions for SED often relies on the availability of strongly labeled audio recordings, where the annotation includes the onset, offset and source of every event. Generating such precise annotations manually is very time consuming, and as a result existing datasets for SED with strong labels are scarce and limited in size. To address this issue, we present Scaper, an open-source library for soundscape synthesis and augmentation. Given a collection of iso-lated sound events, Scaper acts as a high-level sequencer that can generate multiple soundscapes from a single, probabilistically defined, “specification”. To increase the variability of the output, Scaper supports the application of audio transformations such as pitch shifting and time stretching individually to every event. To illustrate the potential of the library, we generate a dataset of 10,000 sound-scapes and use it to compare the performance of two state-of-the-art algorithms, including a breakdown by soundscape characteristics. We also describe how Scaper was used to generate audio stimuli for an audio labeling crowdsourcing experiment, and conclude with a discussion of Scaper's limitations and potential applications.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, methodology): “URBAN-SED [76] is a dataset synthesized by mixing sound events from the 10 classes of UrbanSound8K”
Sound Event Detection Using Point-Labeled Data (2019, methodology): “We evaluate models on URBAN-SED dataset which contains 10,000 soundscapes generated using the Scaper soundscape synthesis library [14].”
USM-SED - A Dataset for Polyphonic Sound Event Detection in Urban Sound Monitoring Scenarios (2021, background): “published in [14] the Scaper library for soundscape synthesis, which was adapted in recent works on SED dataset generation [15].”
Sound Event Detection in Synthetic Domestic Environments (2020, unknown): “We generated strongly annotated synthetic soundscapes using the Scaper library [11].”
The Benefit of Temporally-Strong Labels in Audio Event Classification (2021, methodology): “Examples include URBAN-SED [5] which is synthesized by mixing sound events from the 10 classes of UrbanSound8K [6] using the Scaper library.”
Robust sound event detection in bioacoustic sensor networks (2019, background): “nsform each sample. For instance, mixing each audio clip in the sensor at hand with a negative (noisy) audio clip belonging to a different sensor in the training set leads to greater generalizability [97]. However, this adaptive procedure causes the number of augmented samples to scale quadratically”
Open-Source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research (2019, methodology): “Modern frameworks such as MUDA [16] and Scaper [18] achieve data provenance by embedding the generation/augmentation parameters within the generated objects, thereby facilitating reproducibility.”
SONYC Urban Sound Tagging (SONYC-UST): A Multilabel Dataset from an Urban Acoustic Sensor Network (2019, background): “The freeﬁeld1010 [6], UrbanSound [7], UrbanSound8k, [7], and Urban-SED [8] datasets contain recordings curated from Freesound [9] rather than recorded in a realistic noise monitoring scenario.”
Improving Sound Event Detection in Domestic Environments using Sound Separation (2020, methodology): “The synthetic soundscapes are generated using Scaper [16].”
What’s all the Fuss about Free Universal Sound Separation Data? (2020, methodology): “The mixing procedure is implemented using Scaper [22] 3 .”
Computational bioacoustics with deep learning: a review and roadmap (2021, background): “Simulation, often involving composing soundscapes from a library of sound clips, has been found useful in urban and domestic sound analysis (Salamon et al., 2017b; Turpault et al., 2021).”
Sound Event Detection in Domestic Environments with Weakly Labeled Data and Soundscape Synthesis (2019, unknown): “The subset of synthetic soundscapes is comprised of 10 second audio clips generated with Scaper [13], a python library for sound-scape synthesis and augmentation.”
SONYC (2019, background): “27 Given a collection of isolated sound events, it functions as a high-level sequencer that can generate multiple sound-scapes from a single probabilistically defined “specification.””