NeMo: a toolkit for building AI applications using Neural Modules

By Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook and 4 more. 2019

411 citations recorded by the source corpus.

NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, conceptual blocks of neural networks that take typed inputs and produce typed outputs. Such modules typically represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system. The toolkit comes with extendable collections of pre-built modules for automatic speech recognition and natural language processing. Furthermore, NeMo provides built-in support for distributed training and mixed precision on latest NVIDIA GPUs. NeMo is open-source this https URL

Subfields: Artificial Intelligence, Speech & Audio, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In ML Systems & Hardware

Citations in the corpus

Robust Speech Recognition via Large-Scale Weak Supervision (2022, methodology): “The results are summarized in Figure 6, showing the distribution of word error rates from Whisper and the 4 commercial ASR services, as well as the NVIDIA STT Conformer-CTC Large model from the NeMo toolkit (Kuchaiev et al., 2019) which performed the best among the open-source models.”
Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “, 2020) from NEMO toolkit (Kuchaiev et al., 2019), which is pre-trained using LibriSpeech.”
Generative Spoken Dialogue Language Modeling (2022, methodology): “4 For the training of HuBERT and HiﬁGAN models, we follow the preprocessing steps of Kuchaiev et al. (2019) 5 to obtain a collection of single-channel voice segments of the Fisher dataset.”
BigVGAN: A Universal Neural Vocoder with Large-Scale Training (2022, methodology): “We use conformer_transducer_large model, where the pretrained checkpoints for the considered languages are publicly available from the NVIDIA NeMo (Kuchaiev et al., 2019) toolkit.”
Hi-Fi Multi-Speaker English TTS Dataset (2021, methodology): “The code for normalization, segmentation and interactive data analysis is publicly available via NeMo toolkit [34].”
TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction (2021, methodology): “The model, the training recipes, and audio samples will be released as part of the NeMo toolkit [31].”
ESPnet-ST: All-in-One Speech Translation Toolkit (2020, neutral): “, 2018) (Kuchaiev et al., 2019) (Zeyer et al.”
Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions (2019, methodology): “We have released the source code and pre-trained models in the NeMo toolkit [3].”
TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model (2020, methodology): “The model, the training recipes, and audio samples will be released as part of the NeMo toolkit [21].”
PHONEME-LEVEL BERT FOR ENHANCED PROSODY OF TEXT-TO-SPEECH WITH GRAPHEME PREDICTIONS (2023, methodology): “The texts are normalized to match the pronunciations for each word using NeMo [13].”
ChipNeMo: Domain-Adapted LLMs for Chip Design (2023, methodology): “All model training procedures are conducted using the NVIDIA NeMo framework [25], incorporating techniques such as tensor parallelism [26] and flash attention [27] for enhanced efficiency.”
WavThruVec: Latent speech representation as intermediate features for neural speech synthesis (2022, methodology): “We collected only audio recordings in English and performed the following preprocessing steps: (1) each sample was processed using the NeMo toolkit [29] for voice activity detection to discard unspoken fragments, (2) followed by speaker diarization to ensure there is a single speaking person per cli”
SPGISpeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition (2021, methodology): “The Conformer-CTC model, provided through the NeMo toolkit [32], is a CTC-based variant of the Conformer which has the same encoder but uses CTC loss [33] instead of RNNT [34].”
SpeechBrain: A General-Purpose Speech Toolkit (2021, background): “With the advent of general-purpose deep learning libraries like TensorFlow [6] and PyTorch [7], more ﬂexible speech recognition frameworks have quickly appeared, e.g., DeepSpeech [8], RETURNN [9], PyTorch-Kaldi [10], Espresso [11], Lingvo [12], Fairseq [13], ESPnet [14], and NeMo [15].”