Jasper: An End-to-End Convolutional Neural Acoustic Model

By Jason Li, Vitaly Lavrukhin, Boris Ginsburg, Ryan Leary, Oleksii Kuchaiev, Jonathan M. Cohen, Huyen Nguyen, Ravi Teja Gadde. 2019

288 citations recorded by the source corpus.

In this paper, we report state-of-the-art results on LibriSpeech among end-to-end speech recognition models without any external training data. Our model, Jasper, uses only 1D convolutions, batch normalization, ReLU, dropout, and residual connections. To improve training, we further introduce a new layer-wise optimizer called NovoGrad. Through experiments, we demonstrate that the proposed deep architecture performs as well or better than more complex choices. Our deepest Jasper variant uses 54 convolutional layers. With this architecture, we achieve 2.95% WER using a beam-search decoder with an external neural language model and 3.86% WER with a greedy decoder on LibriSpeech test-clean. We also report competitive results on the Wall Street Journal and the Hub5'00 conversational evaluation datasets.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, methodology): “…to compare results without the use of data augmentation, since different authors use different data augmentation methods (e.g., speed perturbation (Li et al., 2019), tempo/volume perturbation (Tuske et al., 2019), SpecAug-ment (Park et al., 2019 We built strong CTC baselines with the architecture…”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “ResNets have been introduced first in the context of computer vision [20], and then successfully applied to speech recognition [21, 22, 23].”
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, background): “The papers corresponding to each data point have been collected in the bibliography [6, 8, 9, 13–21].”
NeMo: a toolkit for building AI applications using Neural Modules (2019, methodology): “As an example, we describe the steps necessary to train a Jasper-like ASR model[14] using nemo_asr.”
Conformer: Convolution-augmented Transformer for Speech Recognition (2020, background): “Recurrent neural networks (RNNs) have been the defacto choice for ASR [1, 2, 3, 4] as they can model the temporal dependencies in the audio sequences effectively [5].”
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, methodology): “Convolution Neural Network (CNN) based models for end-toend (E2E) speech recognition is attracting an increasing amount of attention [1, 2, 3, 4].”
Masked Language Model Scoring (2019, background): “This difference manifests with self-attentive LMs like SANLMs and Transformer-XL (Dai et al., 2019), as recently explored for N -best rescoring in ASR (Li et al., 2019; Shin et al., 2019).”
One TTS Alignment to Rule Them All (2021, methodology): “We synthesize speech with models trained on LJ Speech, and recognize it with Jasper [22].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025, unknown): “This momentum, as visualized in Figure 1, highlights the significance of recent architectural advances in scaling autonomous agents for real-world deployment.”
High Fidelity Speech Regeneration with Application to Speech Enhancement (2021, methodology): “In our implementation, we use the public implementation [20] of Wav2Letter [17].”
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN (2020, methodology): “For the evaluation of SLT simultaneity, we force-aligned words from the transcript to the sound using a model trained with Jasper (Li et al., 2019) and resorted to fully manual identiﬁcation of word boundaries in the few ﬁles where forced alignment failed.”
Adaptive Knowledge Distillation Based on Entropy (2020, result): “To verify the performance of pre-trained models, we compared the results with the recent papers [17] [18] [19].”
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks (2019, methodology): “We evaluated NovoGrad on the following models: • ResNet-50 [6] — for ImageNet classification • Transformer-big [31] — for WMT 2014 English-to-German translation • Jasper [12] — for LibriSpeech speech recognition • Transformer-XL [2] — for WikiText-103 word-level language modeling and compared it to ”
Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription (2020, background): “LibriSpeech [6]), end-to-end models’ performance is great [7, 8, 9].”
Mellotron: Multispeaker Expressive Voice Synthesis by Conditioning on Rhythm, Pitch and Global Style Tokens (2019, unknown): “First, we extract text information from an audio ﬁle by either using an automatic speech recognition model [11, 12] or by manually transcribing the text.”
TTS Skins: Speaker Conversion via ASR (2019, methodology): “In our method, the encoder is taken to be a subnetwork of a pre-trained ASR network, specifically the public implementation [21] of Wav2Letter [22].”
A Comparison of Transformer and LSTM Encoder Decoder Models for ASR (2019, methodology): “The Transformer LM for ASR has been used in [21, 33].”
From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition (2019, unknown): “9 NVIDIA Jasper [49] 333M 6g 3.”