Singing Voice Separation with Deep U-Net Convolutional Networks

By Andreas Jansson, Eric J. Humphrey, Nicola Montecchio, Rachel M. Bittner, Aparna Kumar, Tillman Weyde. 2017

462 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Music Source Separation in the Waveform Domain (2019, methodology): “A convolutional network with a U-Net structure called Wave-U-Net was used ﬁrst on spectrograms [Jansson et al., 2017] and then adapted to the waveform domain [Stoller et al., 2018].”
Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation (2018, background): “T in most speech separation systems is at least 32 ms [5], [7], [8] and is even greater in music separation applications, which require an even higher resolution spectrogram (higher than 90 ms) [15], [16]. Because these issues arise from formulating the separation problem in the time-frequency domai”
Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation (2018, methodology): “• We replace strided transposed convolution used in previous work [7, 16] for upsampling feature maps with linear interpolation followed by a normal convolution to avoid artifacts.”
Spleeter: a fast and efficient music source separation tool with pre-trained models (2020, methodology): “The pre-trained models are U-nets (Jansson et al., 2017) and follow similar specifications as in (Prétet, Hennequin, Royo-Letelier, & Vaglio, 2019).”
Improved Speech Enhancement with the Wave-U-Net (2018, unknown): “Recently, the U-Net architecture on magnitude spectrograms has The Wave-U-Net architecture following [12]. achieved new state of the art results in audio source separation for music [6] and speech dereverbration [2].”
Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective (2018, methodology): “We use a 4-layer BLSTM with convolutional encoder-decoder structures and skip connections [31], [32] in the chimera++ network (see Fig.”
Singing Voice Separation: A Study on Training Data (2019, methodology): “The main modiﬁcation compared to [5] was to integrate stereo processing: we used 3D tensors (channels, time steps, frequency bins) as input and output of the network.”
Harp-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Coding (2021, unknown): “The Hypothetical Codec with the U-Net Architecture In the original U-Net [14] and its variants for audio processing [15, 18] skip connections pair up the encoder and decoder layers through identity shortcuts to improve the AE’s reconstruction performance.”
Synthesizing Audio with Generative Adversarial Networks (2018, background): “Jansson et al. (2017); Michelsanti & Tan (2017); Donahue et al. (2018) all use GANs in combination with unstructured losses to map spectrograms in one domain to spectrograms in another.”
Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, background): “ing: Partly motivated by the recent success of DenseNet [16] and UNet [32] in music source separation [19], [35], we propose a Dense-UNet structure for simultaneous grouping.”
Computational bioacoustics with deep learning: a review and roadmap (2021, methodology): “DL methods for this are based on denoising autoencoders and/or more recently the u-net , which is a specialised CNN architecture for mapping back to the same domain (Jansson et al., 2017).”
Mmdenselstm: An Efficient Combination of Convolutional and Recurrent Neural Networks for Audio Source Separation (2018, methodology): “To address this problem, a multi-scale structure was used to adapt a CNN to solve the SS problem in [17,18], where convolutional layers were applied on multiple scales obtained by downsampling feature maps, and low-resolution feature maps were progressively upsampled to recover the original…”
HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement (2022, methodology): “Note that similar techniques have been used in speech enhancement literature as a standalone solution (Wisdom et al., 2019; Jansson et al., 2017).”
Denoising Auto-Encoder with Recurrent Skip Connections and Residual Regression for Music Source Separation (2018, methodology): “Symmetric skip connections have been used for biomedical image segmentation [2] and singing voice separation [3].”
Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms (2018, unknown): “Existing research has looked into techniques for speech separation [22] and speech enhancement [12] as well as separating music into its instrumental components [13] or removing the vocals to produce a karaoke version of the track [9].”