Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding

By Kai Wei, Thanh Tran, Feng-Ju Chang, Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Jing Liu, Anirudh Raju, Ross McGowan, Nathan Susanj, A. Rastrow and 1 more. 2021

10 citations recorded by the source corpus.

Recent years have seen significant advances in end-to-end (E2E) spoken language understanding (SLU) systems, which directly predict intents and slots from spoken audio. While dialogue history has been exploited to improve conventional text-based natural language understanding systems, current E2E SLU approaches have not yet incorporated such critical contextual signals in multi-turn and task-oriented dialogues. In this work, we propose a contextual E2E SLU model ar-chitecture that uses a multi-head attention mechanism over encoded previous utterances and dialogue acts (actions taken by the voice assistant) of a multi-turn dialogue. We detail alternative methods to integrate these contexts into the state-of-the-art recurrent and transformer-based models. When applied to a large de-identified dataset of utterances collected by a voice assistant, our method reduces average word and seman-tic error rates by 10.8% and 12.6%, respectively. We also present results on a publicly available dataset and show that our method significantly improves performance over a non-contextual baseline.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tara N. Sainath
2. Bo Li
3. Jinyu Li
4. Trevor Strohman
5. Dong Yu
6. Daniel Povey
7. Li Deng
8. Shuo-yiin Chang

Citations in the corpus

• Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context (2023, background): “Contextualized speech recognition utilizes the cross-utterance context to improve the accuracy of ASR systems and yields promising results [12, 13].”
• Context-Aware end-to-end ASR Using Self-Attentive Embedding and Tensor Fusion (2023, methodology): “To build the end-to-end architectures, we inject context into RNN-T decoder in contrast to the previous works using encoder [15,16] or other E2E architectures e.g. CTC/Attention [13, 14] or LAS [12].”