Context-Aware end-to-end ASR Using Self-Attentive Embedding and Tensor Fusion

By Shuo-yiin Chang, Chaoyang Zhang, Tara N. Sainath, Bo Li, Trevor Strohman. 2023

16 citations recorded by the source corpus.

Typical automatic speech recognition (ASR) systems are built to recognize independent utterances without using the cross-utterance context. However, the context over multiple utterances often provides useful information for the ASR task. In this work, we propose a context-aware end-to-end ASR model that injects the self-attentive context embedding into the decoder of the recurrent neural network transducer (RNN-T). We also propose a factorised 3-way tensor fusion approach to fuse the context embedding with the acoustic representations extracted from the acoustic encoder and the text representations obtained using the prediction network based on the previous subword units. Experimental results on a long-form Youtube ASR task shows that the proposed approach achieves 10.8% relative word error rate reductions.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zengwei Yao
2. Liyong Guo
3. Xiaoyu Yang
4. Wei Kang
5. Fangjun Kuang
6. Long Lin
7. Yifan Yang
8. Daniel Povey

In Signal Processing

1. Zengwei Yao
2. Liyong Guo
3. Xiaoyu Yang
4. Wei Kang
5. Fangjun Kuang
6. Long Lin
7. Yifan Yang
8. Daniel Povey

Citations in the corpus

• Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context (2023, background): “Contextualized speech recognition utilizes the cross-utterance context to improve the accuracy of ASR systems and yields promising results [12, 13].”