Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context

By Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, Daniel Povey. 2023

143 citations recorded by the source corpus.

In this paper, we introduce Libriheavy, a large-scale ASR corpus consisting of 50,000 hours of read English speech derived from LibriVox. To the best of our knowledge, Libriheavy is the largest freely-available corpus of speech with supervisions. Different from other open-sourced datasets that only provide normalized transcriptions, Libriheavy contains richer information such as punctuation, casing and text context, which brings more flexibility for system building. Specifically, we propose a general and efficient pipeline to locate, align and segment the audios in previously published Librilight to its corresponding texts. The same as Librilight, Libriheavy also has three training subsets small, medium, large of the sizes 500h, 5000h, 50000h respectively. We also extract the dev and test evaluation sets from the aligned audios and guarantee there is no overlapping speakers and books in training sets. Baseline systems are built on the popular CTC-Attention and transducer models. Additionally, we open-source our dataset creatation pipeline which can also be used to other audio alignment tasks.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Jinyu Li
3. Lei He
4. Shujie Liu
5. Sheng Zhao
6. Zhuo Chen
7. Tao Qin
8. Yanqing Liu

In Evaluation & Benchmarks

1. Zhuo Chen
2. Jinyu Li
3. Sanyuan Chen
4. Shujie Liu
5. Yanqing Liu
6. Zi-Hua Zhang
7. Ziyang Ma
8. Chengyi Wang

In Speech & Audio

1. Lei He
2. Xu Tan
3. Sheng Zhao
4. Yanqing Liu
5. Jinyu Li
6. Shujie Liu
7. Zhuo Chen
8. Sanyuan Chen

In Signal Processing

1. Jinyu Li
2. Lei He
3. Xu Tan
4. Sheng Zhao
5. Shujie Liu
6. Zhuo Chen
7. Yu Wu
8. Sanyuan Chen

Citations in the corpus

• Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (2023, positive): “We train VALL-E with Libriheavy corpus [15], a labeled version of Librilight corpus [16] that consists 50 k hours of English speech derived from open-source audio books with over 7000 unique speakers.”
• VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers (2024, methodology): “We use Libriheavy corpus [Kang et al., 2024] as the training data.”
• BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024, methodology): “We then perform partial text restoration (similar to [56]) by: 1) searching for source texts from the Internet and tying them to each recording; 2) matching the ASR transcript with the source text sentence by sentence, replacing the former with latter if their difference is small enough (length no…”
• F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching (2024, methodology): “The model is trained on Libriheavy [70] 50K hours English dataset.”