Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

By Alex Graves, Santiago Fernández, Faustino J. Gomez, J. Urgen Schmidhuber. 2006

5,319 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

Sequence to Sequence Learning with Neural Networks (2014, methodology): “The Connectionist Sequence Classification is another popular technique for mapping sequences to sequences with neural networks, but it assumes a monotonic alignment between the inputs and the outputs [11].”
Speech recognition with deep recurrent neural networks (2013, methodology): “Instead of combining RNNs with HMMs, it is possible to train RNNs ‘end-to-end’ for speech recognition [8, 9, 10].”
Deep Learning Scaling is Predictable, Empirically (2017, methodology): “We use Adam to optimize connectionist temporal classiﬁcation loss (CTC, Graves et al. (2006)).”
Mixed Precision Training (2017, methodology): “The model used for training on the English dataset consists of two 2D convolution layers, three recurrent layers with GRU cells, 1 row convolution layer and Connectionist temporal classification (CTC) cost layer (Graves et al., 2006).”
Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, methodology): “Conveniently, the log term in Equation 12 can be computed exactly and efﬁciently via dynamic programming (Graves et al., 2006).”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “acoustic models with Connectionist Temporal Classification (CTC) (Graves et al., 2006) or sequence-to-sequence (Seq2Seq) (Sutskever et al.”
End-to-End People Detection in Crowded Scenes (2015, background): “Suitable loss functions have previously been proposed in structured speech recognition and natural language processing [7].”
Multimodal Transformer for Unaligned Multimodal Language Sequences (2019, methodology): “Connectionist Temporal Classification (CTC) (Graves et al., 2006) was first proposed for unsupervised Speech to Text alignment.”
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020, methodology): “on labeled data with a Connectionist Temporal Classification (CTC) loss [14, 4] to be used for downstream speech recognition tasks (§ 3) Previous work learned a quantization of the data followed by a contextualized representations with a self-attention model [5, 4], whereas our approach solves both ”
FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech (2022, methodology): “To build ASR baselines, we add a decoder which consists of two LSTM[27] layers to fine-tune from pre-trained models, using a Connectionist Temporal Classification (CTC) [28] loss.”
mSLAM: Massively multilingual joint pre-training for speech and text (2022, methodology): “…text from 101 languages. mSLAM is a multilingual extension of SLAM (Bapna et al., 2021), with the addition of a Connectionist Temporal Classification (CTC) loss (Graves et al., 2006) on the paired speech-text data, to reduce interference and encourage stronger alignment across the two modalities.”
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale (2021, methodology): “(2020b) by adding a linear layer on top of the pretrained model to predict the output vocabulary and train using Connectionist Temporal Classification (CTC; Graves et al. 2006).”
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units (2021, methodology): “The CTC target vocabulary includes 26 English characters, a space token, an apostrophe, and a special CTC blank symbol.”
Unsupervised Speech Recognition (2021, background): “Such models are often classified into two categories: hybrid (Juang et al., 1986; Young, 1996; Povey, 2005; Hinton et al., 2012; Abdel-Hamid et al., 2012; Bourlard and Morgan, 2012) or end-to-end (Graves et al., 2006; Graves, 2012; Chorowski et al., 2015; Rao et al., 2017).”
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training (2021, methodology): “Pre-trained models are fine-tuned with connetionist temporal classification [40].”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, methodology): “2 with CTC criterion (Graves et al., 2006) and wav2vec 2.”
MLS: A Large-Scale Multilingual Dataset for Speech Research (2020, methodology): “We chose ASG criterion over Connectionist Temporal Classification [9] (CTC) criterion since ASG doesn’t exhibit delay in transcriptions compared to CTC [10].”
Self-Training and Pre-Training are Complementary for Speech Recognition (2020, positive): “After pre-training on the unlabeled data, this model is fine-tuned on the labeled data using Connectionist Temporal Classification (CTC; [35]) and a letter-based output vocabulary.”