Wav2Letter: an End-to-End ConvNet-based Speech Recognition System

By Ronan Collobert, Christian Puhrsch, Gabriel Synnaeve. 2016

296 citations recorded by the source corpus.

This paper presents a simple end-to-end model for speech recognition, combining a convolutional network based acoustic model and a graph decoding. It is trained to output letters, with transcribed speech, without the need for force alignment of phonemes. We introduce an automatic segmentation criterion for training from sequence annotation without alignment that is on par with CTC (Graves et al., 2006) while being simpler. We show competitive results in word error rate on the Librispeech corpus (Panayotov et al., 2015) with MFCC features, and promising results from raw waveform.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, background): “Deep architectures trained end-to-end have been very successful in many other domains, such as speech recognition (Hinton et al., 2012; Collobert, Puhrsch, and Synnaeve, 2016), computer vision (Krizhevsky, Sutskever, and Hinton, 2012), and machine translation (Sutskever, Vinyals, and Le, 2014;…”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “, 2016), ASG (Collobert et al., 2016; Zeghidour et al., 2018), LF-MMI (Hadian et al.”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, methodology): “The ASR systems are TDS models (Hannun et al., 2019) trained with ASG criterion (Collobert et al., 2016) on audio tracks from in-house de-identified video data.”
MLS: A Large-Scale Multilingual Dataset for Speech Research (2020, methodology): “For this, we have used trained acoustic models in each of the languages using Time-Depth Separable (TDS) Convolutions[7] with Auto-Segmentation Criterion [8] (ASG) loss.”
Rethinking Evaluation in ASR: Are Our Models Robust Enough? (2020, methodology): “For another kind of transfer in [Kunze et al., 2017], the authors transferred LibriSpeech trained wav2letter [Collobert et al., 2016] models to German by ﬁne-tuning them on German, with better performance than training from scratch.”
Learning Robust and Multilingual Speech Representations (2020, positive): “In order to further investigate how the representations interact with larger speech recognition models, we use the time-delay neural networks ( TDNN ) that are commonly used in speech recognition (Collobert et al., 2016; Kuchaiev et al., 2018).”
Effectiveness of self-supervised pre-training for speech recognition (2019, methodology): “, 2019b) where the a BERT-like model is trained with the masked language model loss, frozen, and then used as a feature extractor in tandem with a final fully supervised convolutional ASR model (Collobert et al., 2016), in this work, our “Discrete BERT” approach achieves an average relative Word Err”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, methodology): “We use wav2letter as accoustic model (Collobert et al., 2016; 2019) and train for 1,000 epochs on 8 GPUs for both TIMIT and WSJ using the auto segmentation criterion.”
Transformers with convolutional context for ASR (2019, background): “Advances in alignment-free sequence-level loss functions like CTC and ASG [3, 4] enabled easier training with letters as output units [5, 6].”
wav2vec: Unsupervised Pre-training for Speech Recognition (2019, positive): “The model is trained using the Auto Segmentation Criterion (ASG; Collobert et al., 2016)) using SGD with momentum.”
Wav2Letter++: A Fast Open-source Speech Recognition System (2018, methodology): “The CTC criterion does not have parameters whereas the ASG and S2S criteria both have parameters which can be learned.”
Learning Filterbanks from Raw Speech for Phone Recognition (2017, methodology): “All our models are trained end-to-end with the Autoseg criterion [20], using stochastic gradient descent.”
Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions (2019, background): “These models have yielded state-of-the-art re-sults in some settings [5], however; approaches such as CRF style end-to-end models [6, 7] and more traditional HMM based models [8] are often superior.”
Fully Convolutional Speech Recognition (2018, methodology): “These acoustic models are trained to predict letters directly with the Auto Segmentation Criterion (ASG) [21].”
Sharp Minima Can Generalize For Deep Nets (2017, background): “…al., 2015; He et al., 2016), machine translation (e.g. Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2015; Wu et al., 2016; Gehring et al., 2016) and speech recognition (e.g. Graves et al., 2013; Hannun et al., 2014; Chorowski et al., 2015; Chan et al., 2016; Collobert et al., 2016).”
A Wavenet for Speech Denoising (2017, methodology): “To this end, discriminative models have been used in an end-to-end learning fashion for speech classification [6, 7, 8].”
Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, methodology): “We took inspiration from wav2letter [7], which uses 1D-convolution layers.”