FurcaNeXt: End-to-end monaural speech separation with dynamic gated dilated temporal convolutional networks

By Ziqiang Shi, Huibin Lin, Liu Liu, Rujie Liu, Jiqing Han. 2019

97 citations recorded by the source corpus.

Deep dilated temporal convolutional networks (TCN) have been proved to be very effective in sequence modeling. In this paper we propose several improvements of TCN for end-to-end approach to monaural speech separation, which consists of 1) multi-scale dynamic weighted gated dilated convolutional pyramids network (FurcaPy), 2) gated TCN with intra-parallel convolutional components (FurcaPa), 3) weight-shared multi-scale gated TCN (FurcaSh), 4) dilated TCN with gated difference-convolutional component (FurcaSu), that all these networks take the mixed utterance of two speakers and maps it to two separated utterances, where each utterance contains only one speaker's voice. For the objective, we propose to train the network by directly optimizing utterance level signal-to-distortion ratio (SDR) in a permutation invariant training (PIT) style. Our experiments on the the public WSJ0-2mix data corpus results in 18.4dB SDR improvement, which shows our proposed networks can leads to performance improvement on the speaker separation task.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Yossi Adi
3. Zhuo Chen
4. Lior Wolf
5. Eliya Nachmani
6. Dong Yu
7. Takuya Yoshioka
8. Jonathan Le Roux

In Signal Processing

1. Jinyu Li
2. Yossi Adi
3. Zhuo Chen
4. Lior Wolf
5. Eliya Nachmani
6. Takuya Yoshioka
7. Jonathan Le Roux
8. Yi Luo

Citations in the corpus

• Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings (2020, methodology): “In this work, we focus on fully supervised voice separation using a single microphone, which has seen a great leap in performance following the recent success of deep learning models considering both frequency domain [1, 2, 7, 8, 9, 10], and time-domain [11, 12, 13, 14, 15, 16] modeling.”
• Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “We compare with the following baseline methods: ADANet (Luo et al., 2018), DPCL++ (Isik et al., 2016), CBLDNN-GAT (Li et al., 2018), TasNet (Luo & Mesgarani, 2018), the Ideal Ratio Mask (IRM), the Ideal Binary Mask (IBM), ConvTasNet (Luo & Mesgarani, 2019), FurcaNeXt (Zhang et al., 2020), and DPRNN ”
• Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation (2019, methodology): “Compared with CNN-based architectures such as temporal convolutional networks (TCNs) that only perform local modeling due to the fixed receptive fields [4, 5, 20], DPRNN is able to fully utilize global information via the inter-chunk RNNs and achieve superior performance with an even smaller model s”
• WHAM!: Extending Speech Separation to Noisy Environments (2019, methodology): “Many of these techniques have surpassed the performance of some noisy phase oracle T-F masks [9,14–16] on the benchmark wsj0-2mix dataset [3].”
• Continuous Speech Separation: Dataset and Analysis (2020, background): “On top of, or aside from, these algorithmic improvements, researchers also sought more effective input features [17, 18] and model architectures [19, 20, 21].”
• Wavesplit: End-to-End Speech Separation by Speaker Clustering (2020, unknown): “(Luo et al., 2018) 10.4 10.8 Grid LSTM PIT (Xu et al., 2018) – 10.2 ConvLSTM-GAT (Li et al., 2018) – 11.0 Chimera++ (Wang et al., 2018b) 11.5 12.0 WA-MISI-5 (Wang et al., 2018c) 12.6 13.1 blstm-TasNet (Luo & Mesgarani, 2018) 13.2 13.6 Conv-TasNet (Luo & Mesgarani, 2019) 15.3 15.6 Conv-TasNet+MBT (La”
• Attention Is All You Need In Speech Separation (2020, unknown): “The field has been revolutionized by the adoption of deep learning techniques [11–16], and with recent works [17–23] achieving impressive results by adopting an end-to-end approach.”
• Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, unknown): “FurcaNeXt [34] integrates gated activations”
• DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement (2020, background): “The time-domain approaches can further fall into two categories — direct regression [2,3] and adaptive front-end approaches [4–6].”