Lhotse: a speech data representation library for the modern deep learning ecosystem

By Piotr Żelasko, Daniel Povey, Jan "Yenda" Trmal, Sanjeev Khudanpur. 2021

50 citations recorded by the source corpus.

Speech data is notoriously difficult to work with due to a variety of codecs, lengths of recordings, and meta-data formats. We present Lhotse, a speech data representation library that draws upon lessons learned from Kaldi speech recognition toolkit and brings its concepts into the modern deep learning ecosystem. Lhotse provides a common JSON description format with corresponding Python classes and data preparation recipes for over 30 popular speech corpora. Various datasets can be easily combined together and re-purposed for different tasks. The library handles multi-channel recordings, long recordings, local and cloud storage, lazy and on-the-fly operations amongst other features. We introduce Cut and CutSet concepts, which simplify common data wrangling tasks for audio and help incorporate acoustic context of speech utterances. Finally, we show how Lhotse leverages PyTorch data API abstractions and adopts them to handle speech data for deep learning.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zengwei Yao
2. Liyong Guo
3. Xiaoyu Yang
4. Wei Kang
5. Fangjun Kuang
6. Long Lin
7. Yifan Yang
8. Daniel Povey

In Signal Processing

1. Zengwei Yao
2. Liyong Guo
3. Xiaoyu Yang
4. Wei Kang
5. Fangjun Kuang
6. Long Lin
7. Yifan Yang
8. Daniel Povey

Citations in the corpus

• Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context (2023, methodology): “We save the metadata of the dataset as Lhotse [15] cuts in JSON lines.”
• Fast and Parallel Decoding for Transducer (2022, unknown): “Lhotse [12] is used for data preparation.”