Recursive speech separation for unknown number of speakers

By Naoya Takahashi, Sudarsanam Parthasaarathy, Nabarun Goswami, Yuki Mitsufuji. 2019

94 citations recorded by the source corpus.

In this paper we propose a method of single-channel speaker-independent multi-speaker speech separation for an unknown number of speakers. As opposed to previous works, in which the number of speakers is assumed to be known in advance and speech separation models are specific for the number of speakers, our proposed method can be applied to cases with different numbers of speakers using a single model by recursively separating a speaker. To make the separation model recursively applicable, we propose one-and-rest permutation invariant training (OR-PIT). Evaluation on WSJ0-2mix and WSJ0-3mix datasets show that our proposed method achieves state-of-the-art results for two- and three-speaker mixtures with a single model. Moreover, the same model can separate four-speaker mixture, which was never seen during the training. We further propose the detection of the number of speakers in a mixture during recursive separation and show that this approach can more accurately estimate the number of speakers than detection in advance by using a deep neural network based classifier.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Shinji Watanabe
3. Zhuo Chen
4. Yossi Adi
5. Shujie Liu
6. Takuya Yoshioka
7. Yu Wu
8. Lior Wolf

In Signal Processing

1. Jinyu Li
2. Shinji Watanabe
3. Zhuo Chen
4. Yossi Adi
5. Takuya Yoshioka
6. Shujie Liu
7. Yu Wu
8. Lior Wolf

Citations in the corpus

• Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings (2020, background): “The authors of [17, 18, 19] suggest to separate one speaker at a time using a recursive solution.”
• Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “Related to our goal of improving the performance on multiple speakers, (Takahashi et al., 2019) introduce a recursive method for speaker separation, based on the Conv-Tasnet model.”
• Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals (2020, methodology): “Compared with the RSAN [18] and OR-PIT [39] as mentioned in Section 4, which also could handle variable number of sources, our model achieves significant performance improvements as a result of our end-to-end speech transduction in time domain.”
• Encoder-Decoder Based Attractor Calculation for End-to-End Neural Diarization (2021, methodology): “OR-PIT is time domain speech separation by which residual output is determined as a mixture that contains other speakers rather than the target speaker.”
• Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors (2021, background): “The first approach is one-and-rest permutation invariant training (OR-PIT) [45], which aims to split a mixture into the one-speaker waveform and the mixture of the rest of the speakers as residual output.”
• Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation (2020, background): “ance. To overcome this problem, time-domain approach is proposed in paper [14] , which directly model the mixture waveform using an encodedecoder framework and has made great progress in recent years [15, 16, 17, 18, 19, 20, 21]. However, the dominant speech separation models are usually based on re”
• End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors (2020, methodology): “One series of methods involve applying the one-vs-rest approach iteratively [18, 19, 20, 21].”
• Meta-Learning Extractors for Music Source Separation (2020, background): “A major design choice in music source separation models is whether to (1) train a separate model for each instrument [12], (2) to use a single class-conditional model, or (3) to use an instrument agnostic approach [16].”
• Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, background): “Various followup studies have been reported, including exploration of the different architectures [3, 4, 5] and recipes [6, 7, 8], extension to multi-channel processing [9, 10] and joint modeling with other tasks such as automatic speech recognition (ASR) [11, 12].”