Fully Convolutional Speech Recognition

By Neil Zeghidour, Qiantong Xu, Vitaliy Liptchinsky, Nicolas Usunier, Gabriel Synnaeve, Ronan Collobert. 2018

98 citations recorded by the source corpus.

Current state-of-the-art speech recognition systems build on recurrent neural networks for acoustic and/or language modeling, and rely on feature extraction pipelines to extract mel-filterbanks or cepstral coefficients. In this paper we present an alternative approach based solely on convolutional neural networks, leveraging recent advances in acoustic models from the raw waveform and language modeling. This fully convolutional approach is trained end-to-end to predict characters from the raw waveform, removing the feature extraction step altogether. An external convolutional language model is used to decode words. On Wall Street Journal, our model matches the current state-of-the-art. On Librispeech, we report state-of-the-art performance among end-to-end models, including Deep Speech 2 trained with 12 times more acoustic data and significantly more linguistic data.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “, 2016), ASG (Collobert et al., 2016; Zeghidour et al., 2018), LF-MMI (Hadian et al.”
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation (2019, methodology): “[8] end-to-end GCNN chars words GCNN 3.”
Iterative Pseudo-Labeling for Speech Recognition (2020, positive): “External Language Model (LM) In modern ASR systems, in addition to the acoustic model pθ , a decoding procedure (typically either WFST-based [22] or beam-search-based (BS) [23, 24]) integrates an external language model (LM).”
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context (2020, background): “Convolution Neural Network (CNN) based models for end-toend (E2E) speech recognition is attracting an increasing amount of attention [1, 2, 3, 4].”
Transformers with convolutional context for ASR (2019, neutral): “amLM 4.79 14.31 4.82 15.30 Gated Cnv[24] 4gramLM 4.6 13.8 4.8 14.5 Tsf w/sin pos embd[13] 4gramLM - - 4.8 13.1 TDS Cnv[25] 4gramLM 3.75 10.70 4.21 11.87 LSTM[23] LSTMLM 3.54 11.52 3.82 12.76 Fully Cnv[26] ConvLM 3.16 10.05 3.44 11.24 TDS Cnv[25] ConvLM 3.01 8.86 3.28 9.84 TDS Cnv[25] None 5.04 14.45”
Self-Training for End-to-End Speech Recognition (2019, methodology): “We train a word piece convolutional LM (ConvLM) using the same model architecture and training recipe as [18].”
Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions (2019, methodology): “The first is a 4-gram trained with KenLM [29] and the second is a convolutional LM (ConvLM) [30] using the same model architecture and training strategy as [31].”
Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion? (2019, neutral): “The resulting WER is very close to the state of the art [38] when trained on train-960.”
Semi-Supervised Speech Recognition via Local Prior Matching (2020, methodology): “We use the same 5,000 word-piece vocabulary for the LM which is trained with the same model configuration and recipe as Zeghidour et al. (2018).”
Scaling Up Online Speech Recognition Using ConvNets (2020, methodology): “Apart from the existing performance optimizations included in wav2letter++ decoder [10], we introduce two further pruning techniques.”
End-to-End Speech Recognition: A Survey (2023, methodology): “4) ConvLM: Convolutional neural networks (ConvLM) have also been applied to LMs [291], [292], [293].”
Who Needs Words? Lexicon-Free Speech Recognition (2019, background): “However, most competitive ASR systems, character based or not, use a beam search decoder constrained on a word-level language model and lexicon [14, 15, 16, 17].”
Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions (2019, unknown): “wav2letter++ [27] speed perturb ConvLM 3.”