Letter-Based Speech Recognition with Gated ConvNets

By Vitaliy Liptchinsky, Gabriel Synnaeve, Ronan Collobert. 2017

73 citations recorded by the source corpus.

In this paper we introduce a new speech recognition system, leveraging a simple letter-based ConvNet acoustic model. The acoustic model requires -- only audio transcription for training -- no alignment annotations, nor any forced alignment step is needed. At inference, our decoder takes only a word list and a language model, and is fed with letter scores from the -- acoustic model -- no phonetic word lexicon is needed. Key ingredients for the acoustic model are Gated Linear Units and high dropout. We show near state-of-the-art results in word error rate on the LibriSpeech corpus using log-mel filterbanks, both on the "clean" and "other" configurations.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

MLS: A Large-Scale Multilingual Dataset for Speech Research (2020, methodology): “For English, however, we use pre-trained model from [12] which uses TDS encoder and CTC loss on LibriSpeech [13] and pseudo labels from LibriVox .”
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, neutral): “ et al., (2015) [19] 5.51 13.97 Povey et al., (2016) [29] 4.28 Han et al., (2017) [30] 3.51 8.58 Yang et al. (2018) [31] 2.97 7.50 CTC/ASG Collobert et al., (2016) [32] 7.2 Liptchinsky et al., (2017) [33] 6.7 20.8 4.8 14.5 Zhou et al., (2018) [34] 5.42 14.70 Zeghidour et al., (2018) [35] 3.44 11.24 ”
Libri-Light: A Benchmark for ASR with Limited or No Supervision (2019, neutral): “Supervised systems (LibriSpeech 1000 h) Gated Cnv+4gramLM[20] 4.”
Wav2Letter++: A Fast Open-source Speech Recognition System (2018, methodology): “The wav2letter++ decoder is a beam-search decoder with several optimizations to improve efficiency [14].”
Fully Convolutional Speech Recognition (2018, methodology): “They have more recently been used in end-to-end frameworks, but only in combination with recurrent layers [7], or n-gram language models [12], or for phone recognition [13, 14].”
Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, methodology): “[8] improved wav2letter by increasing the model depth to 19 convolutional layers and adding Gated Linear Units (GLU) [9], weight normalization [10] and dropout.”
Improved training of end-to-end attention models for speech recognition (2018, background): “(ASG) [64] can be seen as a variant of the CTC criterion and”
A Fully Differentiable Beam Search Decoder (2019, methodology): “Unlike ASG, CTC assumes output tokens are conditionally independent given the input and includes an optional blank which makes the graph less regular (Liptchinsky et al., 2017).”
Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions (2019, methodology): “This allows us to use much wider kernels, roughly 3 times larger than kernels used in wav2letter [16] or Jasper [14] models.”
Fully Neural Network Based Speech Recognition on Mobile and Embedded Devices (2018, methodology): “Convolutional neural networks (CNNs) have also been used for sequence learning, such as machine translation [18, 19], language modeling [20], and speech recognition [21].”
Who Needs Words? Lexicon-Free Speech Recognition (2019, methodology): “We followed [15] for the architecture choices, picking the ”high dropout” model with 19 convolutional layers for Librispeech, and the lighter version with 17 layers for WSJ.”
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition (2019, methodology): “We addressed the challenges of adapting self-attention to CTC and to speech recognition, showing that SAN-CTC is competitive with or outperforms existing end-to-end models on WSJ and LibriSpeech.”
To Reverse the Gradient or Not: an Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition (2018, methodology): “We set Łacoustic(θr, θy) to be the AutoSeg Criterion (ASG) criterion [16, 17], which is similar to the Connectionist Temporal Classification (CTC) criterion [18].”