Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions

By Samuel Kriman, Stanislav Beliaev, Boris Ginsburg, Jocelyn Huang, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, Yang Zhang. 2019

328 citations recorded by the source corpus.

We propose a new end-to-end neural acoustic model for automatic speech recognition. The model is composed of multiple blocks with residual connections between them. Each block consists of one or more modules with 1D time-channel separable convolutional layers, batch normalization, and ReLU layers. It is trained with CTC loss. The proposed network achieves near state-of-the-art accuracy on LibriSpeech and Wall Street Journal, while having fewer parameters than all competing models. We also demonstrate that this model can be effectively fine-tuned on new datasets.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Speech & Audio

In Signal Processing

Citations in the corpus

Conformer: Convolution-augmented Transformer for Speech Recognition (2020, methodology): “Recurrent neural networks (RNNs) have been the defacto choice for ASR [1, 2, 3, 4] as they can model the temporal dependencies in the audio sequences effectively [5].”
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, methodology): “This is a big improvement over previous CNN based architectures such as QuartzNet[4], and it outperforms transformer and LSTM based models [11, 10, 24, 9, 25].”
Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “For example, TalkNet [12] uses the CTC-based automatic speech recognition (ASR) model Quartznet [111], FastSpeech 2 [174] uses the forcedalignment tool MFA toolkit [135], DurIAN [243] uses a external alignment model [51, 252], RobuTrans [119] uses speech recognition tools, Parallel Tacotron [49] and”
SpeechBrain: A General-Purpose Speech Toolkit (2021, methodology): “Due to its industrial orientation, NeMo offers efficient ready-to-use models, such as Jasper [63], QuartzNet [64], and Citrinet [65].”
Scaling Up Online Speech Recognition Using ConvNets (2020, background): “Other architectures, such as the TimeChannel Separable convolution have also shown similar gains in computational efficiency with little if any hit to accuracy [21].”
TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction (2021, methodology): “The grapheme duration predictor is a 1D time channel separable convolutional NN based on the QuartzNet architecture [8].”
TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model (2020, methodology): “To obtain the ground truth grapheme duration we use QuartzNet15x5 [17] with a minor modification: we set the stride in the first convolutional layer to 1 to make the length of the CTC output equal to the length of the mel-spectrogram.”
Wav2KWS: Transfer Learning From Speech Representations for Keyword Spotting (2021, background): “Although improved end-to-end models [21] (e.g., QuartzNet [35]) for KWS are available, a pretrained model on a large-scale speech corpus has not been used for KWS. Weachieve accurate KWS with a small training dataset even if there are not additional synthesized data.”
Hi-Fi Multi-Speaker English TTS Dataset (2021, methodology): “To segment the original text into chapters that correspond to the chapter level audio recordings, we transcribe audio files with the QuartzNet ASR model [22].”