FaSNet: Low-Latency Adaptive Beamforming for Multi-Microphone Audio Processing

By Yi Luo, Enea Ceolini, Cong Han, Shih-Chii Liu, Nima Mesgarani. 2019

186 citations recorded by the source corpus.

1. ABSTRACT Beamforming has been extensively investigated for multi-channel audio processing tasks. Recently, learning-based beamforming methods, sometimes called neural beamformers, have achieved significant improvements in both signal quality (e.g. signal-to-noise ratio (SNR)) and speech recognition (e.g. word error rate (WER)). Such systems are generally non-causal and require a large context for robust estimation of inter-channel features, which is impractical in applications requiring low-latency responses. In this paper, we propose filter-and-sum network (FaSNet), a time-domain, filter-based beamforming approach suitable for low-latency scenarios. FaSNet has a two-stage system design that first learns frame-level time-domain adaptive beamforming filters for a selected reference channel, and then calculate the filters for all remaining channels. The filtered outputs at all channels are summed to generate the final output. Experiments show that despite its small model size, FaSNet is able to outperform several traditional oracle beamformers with respect to scale-invariant signal-to-noise ratio (SI-SNR) in reverberant speech enhancement and separation tasks. Moreover, when trained with a frequency-domain objective function on the CHiME-3 dataset, FaSNet achieves 14.3% relative word error rate reduction (RWERR) compared with the baseline model. These results show the efficacy of FaSNet particularly in reverberant and noisy signal conditions.

Subfields: Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Lior Wolf
2. Yossi Adi
3. Eliya Nachmani
4. Zhuo Chen
5. Takuya Yoshioka
6. Yi Luo
7. Nima Mesgarani

In Signal Processing

1. Yossi Adi
2. Lior Wolf
3. Eliya Nachmani
4. Nima Mesgarani
5. Zhuo Chen
6. Takuya Yoshioka
7. Yi Luo
8. Jonathan Le Roux

Citations in the corpus

• Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “Recent studies pointed out the importance of choosing small kernel size for the encoder (Luo et al., 2019c).”
• End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation (2019, methodology): “The original FasNet has significantly better performance than all TasNet models, which matches the observation in [13].”