End-to-end music source separation: is it possible in the waveform domain?

By Francesc Lluís, Jordi Pons, Xavier Serra. 2018

78 citations recorded by the source corpus.

Most of the currently successful source separation techniques use the magnitude spectrogram as input, and are therefore by default omitting part of the signal: the phase. To avoid omitting potentially useful information, we study the viability of using end-to-end models for music source separation --- which take into account all the information available in the raw audio signal, including the phase. Although during the last decades end-to-end music source separation has been considered almost unattainable, our results confirm that waveform-based models can perform similarly (if not better) than a spectrogram-based deep learning model. Namely: a Wavenet-based model we propose and Wave-U-Net can outperform DeepConvSep, a recent spectrogram-based deep learning model.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Ming-Hsuan Yang
4. Trevor Darrell
5. Xiaogang Wang
6. Bernt Schiele
7. Cordelia Schmid
8. Jitendra Malik

In Natural Language Processing

1. Kexin Zhao
2. Bryan Catanzaro
3. Zhifeng Kong
4. Jiaji Huang
5. Wei Ping
6. Francis R. Bach
7. Yi Luo
8. Zhuo Chen

In Speech & Audio

1. Alexandre D'efossez
2. Zhuo Chen
3. Yossi Adi
4. Takuya Yoshioka
5. Wei Ping
6. Kexin Zhao
7. Nicolas Usunier
8. Francis R. Bach

In Signal Processing

1. Alexandre D'efossez
2. Yossi Adi
3. Takuya Yoshioka
4. Zhuo Chen
5. Wei Ping
6. Yi Luo
7. Kexin Zhao
8. Lior Wolf

Citations in the corpus

• DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, methodology): “3 The similar architecture has been applied for source separation (Rethage et al., 2018; Lluís et al., 2018).”
• Music Source Separation in the Waveform Domain (2019, methodology): “, 2018] and then adapted to source separation [Lluís et al., 2018].”
• Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation (2019, background): “to the underlying clean signals without an explicit front-end, typically by using some form of one-dimensional convolutional neural networks (1-D CNNs) [2, 7, 10].”
• Meta-Learning Extractors for Music Source Separation (2020, methodology): “Here SDR was either taken from the respective papers ([10][11][12]) or from the SiSEC18 [4] evaluation scores ([13][14][15]), and we show the median of frames, median of tracks, evaluated with BSSEval v4 [4].”