Attention-based Neural Beamforming Layers for Multi-channel Speech Recognition

By Bhargav Pulugundla, Yang Gao, Brian King, Gokce Keskin, Sri Harish Reddy Mallidi, Minhua Wu, Jasha Droppo, Roland Maas. 2021

Attention-based beamformers have recently been shown to be effective for multi-channel speech recognition. However, they are less capable at capturing local information. In this work, we propose a 2D Conv-Attention module which combines convolution neural networks with attention for beamforming. We apply self- and cross-attention to explicitly model the correlations within and between the input channels. The end-to-end 2D Conv-Attention model is compared with a multi-head self-attention and superdirective-based neural beamformers. We train and evaluate on an in-house multi-channel dataset. The results show a relative improvement of 3.8% in WER by the proposed model over the baseline neural beamformer.

Subfields: Model Architectures, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. Mor Geva
2. Jonathan Berant
3. Omer Levy
4. Yonatan Belinkov
5. Alex Andonian
6. David Bau
7. Kevin Meng

In Speech & Audio

1. Jonathan Berant

Citations in the corpus

• Transformer Feed-Forward Layers Are Key-Value Memories (2020, background): “Previous work also high-lighted the importance of feed-forward layers in transformers (Press et al., 2020; Pulugundla et al., 2021; Xu et al., 2020).”