Continuous Speech Separation with Conformer

By Sanyuan Chen, Yu Wu, Zhuo Chen, Jinyu Li, Chengyi Wang, Shujie Liu, M. Zhou. 2020

151 citations recorded by the source corpus.

Continuous speech separation was recently proposed to deal with the overlapped speech in natural conversations. While it was shown to significantly improve the speech recognition performance for multichannel conversation transcription, its effectiveness has yet to be proven for a single-channel recording scenario. This paper examines the use of Conformer architecture in lieu of recurrent neural networks for the separation model. Conformer allows the separation model to efficiently capture both local and global context information, which is helpful for speech separation. Experimental results using the LibriCSS dataset show that the Conformer separation model achieves the state of the art results for both single-channel and multi-channel settings. Results for real meeting recordings are also presented, showing significant performance gains in both word error rate (WER) and speaker-attributed WER.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Shujie Liu
3. Zhuo Chen
4. Dong Yu
5. Yu Wu
6. Zhou Zhao
7. Takuya Yoshioka
8. Sanyuan Chen

In Signal Processing

1. Jinyu Li
2. Zhuo Chen
3. Takuya Yoshioka
4. Zhou Zhao
5. Shujie Liu
6. Yu Wu
7. Jinglin Liu
8. Yi Ren

Citations in the corpus

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, methodology): “WavLM achieves SOTA performance on the speech separation LibriCSS benchmark [20], and significantly outperforms the previous Conformer model [21] by a 27.”
• TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation (2022, background): “The conformer model (Guo et al., 2021; Chen et al., 2021) has demonstrated its efficiency in combining convolution neural networks and transformers to model both local and global dependencies of audio in a parameter-efficient way, achieving state-of-the-art results on various downstream tasks.”
• Personalized speech enhancement: new models and Comprehensive evaluation (2021, background): “Bottleneck Blocks: The success of the transformer and its variants for speech processing [20, 21, 13] inspired the design of the bottleneck layers.”
• Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, methodology): “Following the work in [18], a modified Conformer [28] structure that consists a stack of conformer blocks is used as M(·) for TF-masks estimation.”
• Ultra Fast Speech Separation Model with Teacher Student Learning (2021, methodology): “The multi-head self-attention module is implemented with relative position embedding as [14, 5, 15].”
• Don’t Shoot Butterfly with Rifles: Multi-Channel Continuous Speech Separation with Early Exit Transformer (2020, methodology): “With the success of Transformer model in speech community [10, 11], Transformer [12] and its variants [13] have successfully been applied to this task.”