WeNet: Production Oriented Streaming and Non-Streaming End-to-End Speech Recognition Toolkit

By Zhuoyuan Yao, Di Wu, Xiong Wang, Binbin Zhang, Fan Yu, Chao Yang, Zhendong Peng, Xiaoyu Chen, Lei Xie, X. Lei. 2021

327 citations recorded by the source corpus.

In this paper, we propose an open source, production first, and production ready speech recognition toolkit called WeNet in which a new two-pass approach is implemented to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model. The main motivation of WeNet is to close the gap between the research and the production of E2E speechrecognition models. WeNet provides an efficient way to ship ASR applications in several real-world scenarios, which is the main difference and advantage to other open source E2E speech recognition toolkits. In our toolkit, a new two-pass method is implemented. Our method propose a dynamic chunk-based attention strategy of the the transformer layers to allow arbitrary right context length modifies in hybrid CTC/attention architecture. The inference latency could be easily controlled by only changing the chunk size. The CTC hypotheses are then rescored by the attention decoder to get the final result. Our experiments on the AISHELL-1 dataset using WeNet show that, our model achieves 5.03\% relative character error rate (CER) reduction in non-streaming ASR compared to a standard non-streaming transformer. After model quantification, our model perform reasonable RTF and latency.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context (2023, methodology): “We build the CTC-Attention baseline using the Wenet [19] framework.”
WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition (2021, methodology): “The corpus composition is described in Section 3 and baseline benchmarks built on Kaldi, ESPnet and WeNet toolkits are introduced in Section 4.”
End-to-End Speech Recognition: A Survey (2023, methodology): “Many of the ideas discussed in this paper have been explored by various industry research labs [265], [326], [327], [328], [329], [330], [331], inter alia .”
Recent Advances in End-to-End Automatic Speech Recognition (2021, background): “First of all, great toolkits [56, 227, 252, 297, 299, 310] are indispensable to the fast development of E2E ASR”
Wespeaker: A Research and Production Oriented Speaker Embedding Learning Toolkit (2022, background): “Unlike the mentioned general-purpose speech processing toolkits, Wenet [11, 12] focuses on end-to-end speech recognition and is designed to bridge the gap between the research and deployment.”
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation (2022, methodology): “[25] Z. Yao, D. Wu, X. Wang, B. Zhang, F. Yu, C. Yang, Z. Peng, X. Chen, L. Xie, and X. Lei, “Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit,” in Proc.”