Multi-Scale multi-band densenets for audio source separation

By Naoya Takahashi, Yuki Mitsufuji. 2017

163 citations recorded by the source corpus.

This paper deals with the problem of audio source separation. To handle the complex and ill-posed nature of the problems of audio source separation, the current state-of-the-art approaches employ deep neural networks to obtain instrumental spectra from a mixture. In this study, we propose a novel network architecture that extends the recently developed densely connected convolutional network (DenseNet), which has shown excellent results on image classification tasks. To deal with the specific problem of audio source separation, an up-sampling layer, block skip connection and band-dedicated dense blocks are incorporated on top of DenseNet. The proposed approach takes advantage of long contextual information and outperforms state-of-the-art results on SiSEC 2016 competition by a large margin in terms of signal-to-distortion ratio. Moreover, the proposed architecture requires significantly fewer parameters and considerably less training time compared with other methods.

Subfields: Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Signal Processing

Citations in the corpus

Music Source Separation in the Waveform Domain (2019, methodology): “They produce a mask on the magnitude spectrums for each frame and each source, and the output audio is generated by running an inverse STFT on the masked spectrograms reusing the input mixture phase [Takahashi and Mitsufuji, 2017, Takahashi et al., 2018, Takahashi and Mitsufuji, 2020].”
Singing Voice Separation: A Study on Training Data (2019, methodology): “Recently, a U-Net [5] and several DenseNet-based systems [6] showed impressive performance: for the ﬁrst time, state-of-the-art models performed similarly to oracle systems for the instrumental part [4].”
Denoising Auto-Encoder with Recurrent Skip Connections and Residual Regression for Music Source Separation (2018, methodology): “The result of our model is only second to that of the MMDenseNet [18] and MMDenseLSTM [19] models proposed by Takahashi et al.”
The 2018 Signal Separation Evaluation Campaign (2018, unknown): “They are: UHL3 [28], TAK2 [26], TAK3 [27], TAU [27, 28].”
FullSubNet+: Channel Attention Fullsubnet with Complex Spectrograms for Speech Enhancement (2022, background): “It has been pointed out that local patterns in the spectrogram are often different in each frequency band [15].”
D3Net: Densely connected multidilated DenseNet for music source separation (2020, methodology): “The low-resolution representations are again upsampled to recover the lost resolution while carrying over the global perspective from downsampled layers [4, 5, 7, 10].”
PhaseNet: Discretized Phase Modeling with Deep Neural Networks for Audio Source Separation (2018, methodology): “The PhaseNet architecture was adapted from the MDenseNet architecture proposed in [5].”
Phase-aware Single-stage Speech Denoising and Dereverberation with U-Net (2020, methodology): “As the proposed PHM and loss function are orthogonal to the network structure, we believe that a better performance can be achieved using the variant of U-Net architectures such as [31, 32].”
Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, background): “The DenseNet structure has shown excellent performance in image classification [16] and music source separation [35].”
Harmonic Gated Compensation Network Plus for ICASSP 2022 DNS Challenge (2022, background): “Since the wide-band (WB, 0~8 KHz) is more likely to contain high energies, tonalities and long sustained sounds, while the high-band (HB, 8~24 KHz) tends to have low energies, noise and rapidly decaying sounds [11], the HB and WB spectra are modeled separately [5].”
Mmdenselstm: An Efficient Combination of Convolutional and Recurrent Neural Networks for Audio Source Separation (2018, methodology): “In [18], we proposed a multi-band structure where each band was linked to a single CNN dedicated to particular frequency bands along with a full-band CNN.”