WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition

By Binbin Zhang, Hang Lv, Pengcheng Guo, Qijie Shao, Chao Yang, Lei Xie, Xin Xu, Hui Bu, Xiaoyu Chen, Chenchen Zeng and 2 more. 2021

361 citations recorded by the source corpus.

In this paper, we present WenetSpeech, a multi-domain Mandarin corpus consisting of 10000+ hours high-quality labeled speech, 2400+ hours weakly labeled speech, and about 10000 hours unlabeled speech, with 22400+ hours in total. We collect the data from YouTube and Podcast, which covers a variety of speaking styles, scenarios, domains, topics and noisy conditions. An optical character recognition (OCR) method is introduced to generate the audio/text segmentation candidates for the YouTube data on the corresponding video subtitles, while a high-quality ASR transcription system is used to generate audio/text pair candidates for the Podcast data. Then we propose a novel end-to-end label error detection approach to further validate and filter the candidates. We also provide three manually labelled high-quality test sets along with WenetSpeech for evaluation – Dev for cross-validation purpose in training, Test_Net, collected from Internet for matched test, and Test_Meeting, recorded from real meetings for more challenging mismatched test. Baseline systems trained with WenetSpeech are provided for three popular speech recognition toolkits, namely Kaldi, ESPnet, and WeNet, and recognition results on the three test sets are also provided as benchmarks. To the best of our knowledge, WenetSpeech is the current largest open-source Mandarin speech corpus with transcriptions, which benefits research on production-level speech recognition.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhou Zhao
2. Yi Ren
3. Rongjie Huang
4. Jinglin Liu
5. Ziyue Jiang
6. Zhe Ye
7. Chen Zhang
8. Chunfeng Wang

In Evaluation & Benchmarks

1. Yi Ren
2. Qiang Yang
3. Shiliang Zhang
4. Rongjie Huang
5. Fengpeng Yue
6. Ziyue Jiang
7. Yuxuan Wang
8. Zejun Ma

In Speech & Audio

1. Zhou Zhao
2. Jinglin Liu
3. Yi Ren
4. Rongjie Huang
5. Ziyue Jiang
6. Qiang Yang
7. Zhihao Du
8. Shiliang Zhang

In Signal Processing

1. Yi Ren
2. Rongjie Huang
3. Jinglin Liu
4. Zhou Zhao
5. Ziyue Jiang
6. Qiang Yang
7. Zhihao Du
8. Shiliang Zhang

Citations in the corpus

• FunCodec: A Fundamental, Reproducible and Integrable Open-Source Toolkit for Neural Speech Codec (2023, methodology): “To assess the generalization ability of our generalized model, we evaluate it on the test sets of multiple open-source corpora, including Librispeech [28], aishell-1 [29], aishell-2 [30], Wenet [31] and Gigaspeech [32].”
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias (2023, methodology): “To obtain the speaker information from GigaSpeech and WenetSpeech, we use a released automatic speaker diarization model called pyannote.audio 12 , which achieves DER=11.24% on the VoxConverse dataset and DER=14.09% on the AISHELL-4 dataset.”
• PolyVoice: Language Models for Speech to Speech Translation (2023, methodology): “To address this challenge, discrete units (Hsu et al., 2021) extracted from the speech are used to replace the target text, and then can be synthesized into the speech (Tjandra et al., 2019; Zhang et al., 2021; Lee et al., 2022).”
• VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation (2023, neutral): “The Chinese ASR data are from WenetSpeech (Zhang et al., 2022), a total of 10,000 hours of multi-domain labeled speech.”
• Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling (2023, neutral): “VALL-E X is trained on two large-scale multi-speaker datasets 1 , LibriLight [Kahn et al., 2020] and WenetSpeech [Zhang et al., 2022a], containing about 60,000 hours of English audiobook speech data and 10,000+ hours of multi-domain Chinese ASR data, respectively.”
• Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts (2023, neutral): “GigaSpeech (Chen et al., 2021a) 8.1M 396.9M 9.6k LibriLight (Kahn et al., 2020) 10.0M 652.3M 17.9k LibriTTS (Zen et al., 2019) 375.1k 23.6M 573.2 WenetSpeech (Zhang et al., 2022) 14.0M 325.5M 9.3K AISHELL-3 (Shi et al., 2020) 88.0k 2.2M 85.0 Didi 500 (Guo et al., 2021) 52 prosodic prompt u rhy and u”