Single Channel Speech Separation with Constrained Utterance Level Permutation Invariant Training Using Grid LSTM

By Chenglin Xu, Wei Rao, Xiong Xiao, Chng Eng Siong, Haizhou Li. 2018

73 citations recorded by the source corpus.

Utterance level permutation invariant training (uPIT) technique is a state-of-the-art deep learning architecture for speaker independent multi-talker separation. uPIT solves the label ambiguity problem by minimizing the mean square error (MSE) over all permutations between outputs and targets. However, uPIT may be sub-optimal at segmental level because the optimization is not calculated over the individual frames. In this paper, we propose a constrained uPIT (cuPIT) to solve this problem by computing a weighted MSE loss using dynamic information (i.e., delta and acceleration). The weighted loss ensures the temporal continuity of output frames with the same speaker. Inspired by the heuristics (i.e., vocal tract continuity) in computational auditory scene analysis, we then extend the model by adding a Grid LSTM layer, that we name it as cuPIT-Grid LSTM, to automatically learn both temporal and spectral patterns over the input magnitude spectrum simultaneously. The experimental results show 9.6% and 8.5% relative improvements on WSJ0-2mix dataset under both closed and open conditions comparing with the uPIT baseline.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuzhou Liu
2. Liu Liu
3. Jiqing Han
4. Ziqiang Shi
5. Rujie Liu
6. Deliang Wang
7. Huibin Lin
8. Arun Narayanan

In Signal Processing

1. Yuzhou Liu
2. Liu Liu
3. Jiqing Han
4. Ziqiang Shi
5. Rujie Liu
6. Deliang Wang
7. Huibin Lin
8. Yuxuan Wang

Citations in the corpus

• Wavesplit: End-to-End Speech Separation by Speaker Clustering (2020, unknown): “Permutation-Invariant Training (PIT) avoids clustering and predicts multiple masks directly [21]–[23].”
• Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation (2020, unknown): “2mix in SISNR (dB), SDR (dB) and Model Size Method SI-SNR SDR Model Size DPCL++ [27] 10.8 - 13.6M uPIT-BLSTM-ST [12] - 10.0 92.7M Deep Attractor [10] 10.5 - - ADANet [28] 10.4 10.8 9.1M Grid LSTM PIT [29] - 10.2 - ConvLSTM-GAT [30] - 11.0 - Chimera++ [31] 11.5 12.0 - WA-MISI-5 [32] 12.6 13.1 32.9M B”
• Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, background): “Recent uPIT improvements include new network structure [24], [43] and new training objectives [24].”
• FurcaNeXt: End-to-end monaural speech separation with dynamic gated dilated temporal convolutional networks (2019, methodology): “0 cuPIT-Grid-RD [32] 10.”