End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation

By Yi Luo, Zhuo Chen, Nima Mesgarani, Takuya Yoshioka. 2019

217 citations recorded by the source corpus.

An important problem in ad-hoc microphone speech separation is how to guarantee the robustness of a system with respect to the locations and numbers of microphones. The former requires the system to be invariant to different indexing of the microphones with the same locations, while the latter requires the system to be able to process inputs with varying dimensions. Conventional optimization-based beamforming techniques satisfy these requirements by definition, while for deep learning-based end-to-end systems those constraints are not fully addressed. In this paper, we propose transform-average-concatenate (TAC), a simple design paradigm for channel permutation and number invariant multi-channel speech separation. Based on the filter-and-sum network (FaSNet), a recently proposed end-to-end time-domain beamforming system, we show how TAC significantly improves the separation performance across various numbers of microphones in noisy reverberant separation tasks with ad-hoc arrays. Moreover, we show that TAC also significantly improves the separation performance with fixed geometry array configuration, further proving the effectiveness of the proposed paradigm in the general problem of multi-microphone speech separation.

Subfields: Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Jinyu Li
2. Sanyuan Chen
3. Shujie Liu
4. Yu Wu
5. Zhuo Chen
6. Naoyuki Kanda
7. Takuya Yoshioka
8. Srdjan Kiti'c

In Signal Processing

1. Jinyu Li
2. Yossi Adi
3. Zhuo Chen
4. Shujie Liu
5. Takuya Yoshioka
6. Lior Wolf
7. Sanyuan Chen
8. Yu Wu

Citations in the corpus

• Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “Recent studies pointed out the importance of choosing small kernel size for the encoder (Luo et al., 2019c).”
• A Survey of Sound Source Localization with Deep Learning Methods (2021, methodology): “Ma and Liu [2018] also used a CNN to perform regression, but they used the CPS matrix as an input feature (see Section 5).”
• Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, background): “Various followup studies have been reported, including exploration of the different architectures [3, 4, 5] and recipes [6, 7, 8], extension to multi-channel processing [9, 10] and joint modeling with other tasks such as automatic speech recognition (ASR) [11, 12].”