The 2018 Signal Separation Evaluation Campaign

By Fabian-Robert Stöter, Antoine Liutkus, Nobutaka Ito. 2018

317 citations recorded by the source corpus.

This paper reports the organization and results for the 2018 community-based Signal Separation Evaluation Campaign (SiSEC 2018). This year's edition was focused on audio and pursued the effort towards scaling up and making it easier to prototype audio separation software in an era of machine-learning based systems. For this purpose, we prepared a new music separation database: MUSDB18, featuring close to 10h of audio. Additionally, open-source software was released to automatically load, process and report performance on MUSDB18. Furthermore, a new official Python version for the BSSEval toolbox was released, along with reference implementations for three oracle separation methods: ideal binary mask, ideal ratio mask, and multichannel Wiener filter. We finally report the results obtained by the participants.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “We follow the SiSec Mus evaluation campaign for music separation (Stöter et al., 2018), where we separate the mixtures into the following four categories: (1) drums, (2) bass, (3) other, and (4) vocals.”
SDR – Half-baked or Well Done? (2018, methodology): “Recently, BSS eval v4 was released as a Python implementation [19]: the authors of Version 4 acknowledged the issue with the original bss eval sources, and recommended using bss eval images instead.”
Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation (2018, methodology): “Models M4 and M6 were submitted as STL1 and STL2 to the SiSec campaign [22].”
Hybrid Transformers for Music Source Separation (2022, background): “Since the 2015 Signal Separation Evaluation Campaign (SiSEC) [4], the community of MSS has mostly focused on the task of training supervised models to separate songs into 4 stems: drums, bass, vocals and other (all the other instruments).”
Improving singing voice separation using Deep U-Net and Wave-U-Net with data augmentation (2019, background): “The current stateof-the-art, and winner of SiSEC 2018 [5], is a combination Long-Short Time Memory networks and Dense Convolutional Neural Networks, presented in [6] and use stereo signals.”
Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016 Challenge (2018, background): “Related public evaluation campaigns also include SiSEC challenge on signal separation [4] and the REVERB challenge on reverberant speech processing research [5].”
Open-Unmix - A Reference Implementation for Music Source Separation (2019, methodology): “…user to use the model right away or fine-tune it on user-provided data (Stöter & Liutkus, 2019b, 2019c) Open-Unmix is based on the bi-directional LSTM model from (Uhlich et al., 2017) and we compared it to other separation models that were submitted to the last SiSEC contest (Stöter et al., 2018).”
Spleeter: a fast and efficient music source separation tool with pre-trained models (2020, background): “• 4 stems separation as in SiSec (Stöter, Liutkus, & Ito, 2018) (vocals, bass, drums and other).”
D3Net: Densely connected multidilated DenseNet for music source separation (2020, methodology): “[10] applied a CNN architecture with a dense skip connectivity pattern, called DenseNet [12], to MSS and obtained state-of-the-art results in SiSEC 2018 [13].”
End-to-end music source separation: is it possible in the waveform domain? (2018, methodology): “Interestingly, though, nearly all successful deep learning algorithms use the magnitude spectrogram as input [1, 2, 3] — and are therefore, by default, omitting part of the signal: the phase.”
Meta-Learning Extractors for Music Source Separation (2020, methodology): “Here SDR was either taken from the respective papers ([10][11][12]) or from the SiSEC18 [4] evaluation scores ([13][14][15]), and we show the median of frames, median of tracks, evaluated with BSSEval v4 [4].”
Denoising Auto-Encoder with Recurrent Skip Connections and Residual Regression for Music Source Separation (2018, methodology): “In this subsection, we show the performance of our submissions to SiSEC2018.”
Bootstrapping Single-channel Source Separation via Unsupervised Spatial Clustering on Stereo Mixtures (2018, background): “Source separation systems based on deep learning are currently the most successful methods for separating recordings containing multiple concurrent sounds in underdetermined conditions, that is, where there are fewer channels than sources [1].”
Mmdenselstm: An Efficient Combination of Convolutional and Recurrent Neural Networks for Audio Source Separation (2018, methodology): “We evaluated the proposed method on the DSD100 and MUSDB18 datasets, prepared for SiSEC 2016 [19] and SiSEC 2018 [26], respectively.”
SPLEETER: A FAST AND STATE-OF-THE ART MUSIC SOURCE SEPARATION TOOL WITH PRE-TRAINED MODELS (2019, background): “• 4 stems separation as in SiSec [7] (vocals, bass, drums and other).”
Singing Voice Separation: A Study on Training Data (2019, methodology): “The 2018 edition of the SiSec campaign [4] shows that the best current systems rely on supervised, deep-learning based models.”
A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation (2017, unknown): “Most of the efforts were carried out in the source separation research community, where six international Signal Separation Evaluation Campaigns (SiSEC) have been run since 2007 [284], [308]–[313].”