Open-Unmix - A Reference Implementation for Music Source Separation

By Fabian-Robert Stöter, Stefan Uhlich, Antoine Liutkus, Yuki Mitsufuji. 2019

307 citations recorded by the source corpus.

Music source separation is the task of decomposing music into its constitutive components, e.g., yielding separated stems for the vocals, bass, and drums. Such a separation has many applications ranging from rearranging/repurposing the stems (remixing, repanning, upmixing) to full extraction (karaoke, sample creation, audio restoration). Music separation has a long history of scientific activity as it is known to be a very challenging problem. In recent years, deep learning-based systems-for the first time-yielded high-quality separations that also lead to increased commercial interest. However, until now, no open-source implementation that achieves state-of-the-art results is available. Open-Unmix closes this gap by providing a reference implementation based on deep neural networks. It serves two main purposes. Firstly, to accelerate academic research as Open-Unmix provides implementations for the most popular deep learning frameworks, giving researchers a flexible way to reproduce results. Secondly, we provide a pre-trained model for end users and even artists to try and use source separation. Furthermore, we designed Open-Unmix to be one core component in an open ecosystem on music separation, where we already provide open datasets, software utilities, and open evaluation to foster reproducible research as the basis of future development.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Music Source Separation in the Waveform Domain (2019, background): “We applied the randomized stabilization to Open-Unmix and Conv-Tasnet with no gain, since, as explained in Section 4.4, both are naturally equivariant with respect to initial time shifts.”
SPLEETER: A FAST AND STATE-OF-THE ART MUSIC SOURCE SEPARATION TOOL WITH PRE-TRAINED MODELS (2019, result): “As can be seen, for most metrics Spleeter is competitive with Open-Unmix and especially on SDR for all instruments. vocals bass drums other SDR SIR SAR ISR SDR SIR SAR ISR SDR SIR SAR ISR SDR SIR SAR ISR Spleeter Mask 6.55 15.19 6.44 12.01 5.10 10.01 5.15 9.18 5.93 12.24 5.78 10.50 4.24 7.86 4.63 9.”
Exploring the Best Loss Function for DNN-Based Low-latency Speech Enhancement with Temporal Convolutional Networks (2020, methodology): “er than that with K = 1 in terms of all metrics as was also noted Table 1: Comparison with existing methods on VBD dataset. Method Causality PESQ CSIG CBAK COVL Noisy - 1.97 3.35 2.44 2.63 Open-Unmix [31,32] noncausal 2.39 3.12 3.19 2.73 DFL [13] noncausal 3.86 3.33 3.22 MDPhD [33] noncausal 2.70 3.”
D3Net: Densely connected multidilated DenseNet for music source separation (2020, unknown): “56 UMX [19] 6.”
Meta-Learning Extractors for Music Source Separation (2020, methodology): “A major design choice in music source separation models is whether to (1) train a separate model for each instrument [12], (2) to use a single class-conditional model, or (3) to use an instrument agnostic approach [16].”
Upsampling Artifacts in Neural Audio Synthesis (2020, unknown): “34 - 648M OpenUnmix [31]: spectrogram-based 5.”
Spleeter: a fast and efficient music source separation tool with pre-trained models (2020, result): “…Signal to Interference Ratio (SIR) and source Image to Spatial distortion Ratio (ISR), are presented in the following table compared to Open-Unmix (Stöter, Uhlich, Liutkus, & Mitsufuji, 2019) and Demucs (Défossez, Usunier, Bottou, & Bach, 2019) (only SDR are reported for Demucs since other…”
Hybrid Transformers for Music Source Separation (2022, methodology): “The former includes models like Open-Unmix [11], a biLSTM with fully connected that predicts a mask on the input spectrogram or D3Net [12] which uses dilated convolutional blocks with dense connections.”