Improving Streaming Automatic Speech Recognition with Non-Streaming Model Distillation on Unsupervised Data

By Thibault Doutre, Wei Han, Min Ma, Zhiyun Lu, Chung-Cheng Chiu, Ruoming Pang, Arun Narayanan, Ananya Misra, Yu Zhang, Liangliang Cao. 2020

26 citations recorded by the source corpus.

Streaming end-to-end automatic speech recognition (ASR) models are widely used on smart speakers and on-device applications. Since these models are expected to transcribe speech with minimal latency, they are constrained to be causal with no future context, compared to their non-streaming counterparts. Consequently, streaming models usually perform worse than non-streaming models. We propose a novel and effective learning method by leveraging a non-streaming ASR model as a teacher to generate transcripts on an arbitrarily large data set, which is then used to distill knowledge into streaming ASR models. This way, we scale the training of streaming models to up to 3 million hours of YouTube audio. Experiments show that our approach can significantly reduce the word error rate (WER) of RNN-T models not only on LibriSpeech but also on YouTube data in four languages. For example, in French, we are able to reduce the WER by 16.4% relatively to a baseline streaming model by leveraging a non-streaming teacher model trained on the same amount of labeled data as the baseline.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Li Deng
4. Yifan Gong
5. Zhuo Chen
6. Chin-Hui Lee
7. Zhong Meng
8. Sabato Marco Siniscalchi

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Yifan Gong
4. Li Deng
5. Zhong Meng
6. Zhuo Chen
7. Chin-Hui Lee
8. Sabato Marco Siniscalchi

Citations in the corpus

• Recent Advances in End-to-End Automatic Speech Recognition (2021, methodology): “Another way to circumvent the spike mismatch is to train the streaming E2E model by using the ASR hypotheses generated from decoding the unlabeled data with a non-streaming E2E model [59].”