The Pytorch-kaldi Speech Recognition Toolkit

By Mirco Ravanelli, Titouan Parcollet, Yoshua Bengio. 2018

241 citations recorded by the source corpus.

The availability of open-source software is playing a remarkable role in the popularization of speech recognition and deep learning. Kaldi, for instance, is nowadays an established framework used to develop state-of-the-art speech recognizers. PyTorch is used to build neural networks with the Python language and has recently spawn tremendous interest within the machine learning community thanks to its simplicity and flexibility.The PyTorch-Kaldi project aims to bridge the gap between these popular toolkits, trying to inherit the efficiency of Kaldi and the flexibility of PyTorch. PyTorch-Kaldi is not only a simple interface between these software, but it embeds several useful features for developing modern speech recognizers. For instance, the code is specifically designed to naturally plug-in user-defined acoustic models. As an alternative, users can exploit several pre-implemented neural networks that can be customized using intuitive configuration files. PyTorch-Kaldi supports multiple feature and label streams as well as combinations of neural networks, enabling the use of complex neural architectures. The toolkit is publicly-released along with a rich documentation and is designed to properly work locally or on HPC clusters.Experiments, that are conducted on several datasets and tasks, show that PyTorch-Kaldi can effectively be used to develop modern state-of-the-art speech recognizers.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Multi-Task Self-Supervised Learning for Robust Speech Recognition (2020, positive): “TIMIT and DIRHA experiments are performed with the PyTorch-Kaldi toolkit [35] using a six-layer multi-layer perceptron and a light GRU [36], respectively.”
Connecting Vision and Language with Localized Narratives (2019, background): “We observe that automatic speech recognition [18,1,46] typically results in imperfect transcriptions.”
Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks (2019, methodology): “%) is a competitive performance as well, especially when compared to state-of-the-art results that do not use complex techniques as system combination, speaker adaptation, or multiple steps of lattice rescoring and decoding [37,38,44,45].”
Interpretable Convolutional Filters with SincNet (2018, methodology): “In the spirit of reproducible research, we release the code of SincNet for speaker identification3 and speech recognition4 (under the PyTorch-Kaldi project [57]).”
TTS Skins: Speaker Conversion via ASR (2019, methodology): “re extracted using a p-norm DNN [37], as done by [16]. Finally, an additional PPG based speaker conversion system is trained by using a different set of PPG features, extracted by using pytorch-kaldi [38] framework. The underlying ASR model achieves an accuracy of PER=14.9% on the TIMIT dataset [39]”
Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion (2020, methodology): “This PPG extractor is implemented based on the open-source package, Pytorch-Kaldi [25].”
A Density Ratio Approach to Language Model Fusion in End-to-End Automatic Speech Recognition (2019, background): “For much of the ASR community, this approach still constitutes the state-of-the-art [3, 22, 23].”
SpeechBrain: A General-Purpose Speech Toolkit (2021, background): “With the advent of general-purpose deep learning libraries like TensorFlow [6] and PyTorch [7], more ﬂexible speech recognition frameworks have quickly appeared, e.g., DeepSpeech [8], RETURNN [9], PyTorch-Kaldi [10], Espresso [11], Lingvo [12], Fairseq [13], ESPnet [14], and NeMo [15].”
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers (2021, background): “Traditional systems involve complex pipelines that require feeding mixed-and-matched hand-crafted features into DNNs [42].”
Learning Speaker Representations with Mutual Information (2018, methodology): “We release the code of LIM within the PyTorch-Kaldi toolkit [23].”
Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “However, the systems in [42] used DNN-HMM alignments (along with monophone regularization) compared to GMM-HMM alignments used in this paper.”
Quaternion Recurrent Neural Networks (2018, methodology): “These results compare favorably with the best results obtained so far with architectures that do not integrate access control in multiple memory layers (Ravanelli et al., 2018a).”
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps (2020, methodology): “We use PyTorch (Paszke et al., 2017), the Kaldi speech recognition toolkit (Povey et al., 2011), and the PyTorch-Kaldi toolkit (Ravanelli et al., 2019) for developing PyTorch speech recognition models for all our experiments and evaluations.”