Streaming small-footprint keyword spotting using sequence-to-sequence models

By Yanzhang He, Rohit Prabhavalkar, Kanishka Rao, Anton Bakhtin, Ian McGraw. 2017

96 citations recorded by the source corpus.

We develop streaming keyword spotting systems using a recurrent neural network transducer (RNN-T) model: an all-neural, end-to-end trained, sequence-to-sequence model which jointly learns acoustic and language model components. Our models are trained to predict either phonemes or graphemes as subword units, thus allowing us to detect arbitrary keyword phrases, without any out-of-vocabulary words. In order to adapt the models to the requirements of keyword spotting, we propose a novel technique which biases the RNN-T system towards a specific keyword of interest. Our systems are compared against a strong sequence-trained, connectionist temporal classification (CTC) based “keyword-filler” baseline, which is augmented with a separate phoneme language model. Overall, our RNN-T system with the proposed biasing technique significantly improves performance over the baseline system.

Subfields: Artificial Intelligence, Efficiency & Compression, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Tara N. Sainath
3. Yu Zhang
4. Yonghui Wu
5. Chung-Cheng Chiu
6. Rohit Prabhavalkar
7. Dong Yu
8. Bo Li

In Efficiency & Compression

1. Jinyu Li
2. Tara N. Sainath
3. Rohit Prabhavalkar
4. Chung-Cheng Chiu
5. Yonghui Wu
6. Yu Zhang
7. Jiahui Yu
8. Wei Han

In Speech & Audio

1. Jinyu Li
2. Tara N. Sainath
3. Yu Zhang
4. Chung-Cheng Chiu
5. Yonghui Wu
6. Ruoming Pang
7. Bo Li
8. Rohit Prabhavalkar

In Signal Processing

1. Jinyu Li
2. Tara N. Sainath
3. Yu Zhang
4. Yonghui Wu
5. Chung-Cheng Chiu
6. Rohit Prabhavalkar
7. Ruoming Pang
8. Bo Li

Citations in the corpus

• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, methodology): “ input utterance in the network to reduce the computation while maintaining the accuracy of the model. 2.1. End-to-end Network: CNN-RNN-Transducer Our network is based on the RNN-Transducer framework [18, 19, 20]. The network contains three components: audio encoder on the input utterance, label enc”
• Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss (2020, methodology): “(3), which are LSTMs in conventional RNN-T architectures [10, 12, 11], using the Transformers described above.”
• Streaming End-to-end Speech Recognition for Mobile Devices (2018, background): “, recognizing specific words or phrases) [4, 5, 6, 7], as well as large vocabulary continuous speech recognition (LVCSR) [8, 9].”
• Recent Advances in End-to-End Automatic Speech Recognition (2021, methodology): “[82] to bias a keyword spotting E2E model toward a specific keyword.”
• No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models (2017, methodology): “In [12], the <eow> symbol was required during decoding and was shown to help in identifying the spacing between words.”