Supervised Sequence Labelling with Recurrent Neural Networks

By Alex Graves. 2012

2,895 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Model Architectures

Citations in the corpus

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling (2014, background): “Recurrent neural networks have recently shown promising results in many machine learning tasks, especially when input and/or output are of variable length [see, e.g., Graves, 2012].”
MaskGAN: Better Text Generation via Filling in the ______ (2018, background): “Recurrent Neural Networks (RNNs) (Graves et al., 2012) are the most common generative model for sequences as well as for sequence labeling tasks.”
Speech recognition with deep recurrent neural networks (2013, methodology): “Instead of combining RNNs with HMMs, it is possible to train RNNs ‘end-to-end’ for speech recognition [8, 9, 10].”
Teaching Machines to Read and Comprehend (2015, methodology): “When used for translation, Deep LSTMs [19] have shown a remarkable ability to embed long sequences into a vector representation which contains enough information to generate a full translation in another language.”
Order Matters: Sequence to sequence for sets (2015, neutral): “Neural models with memories coupled to differentiable addressing mechanism have been successfully applied to handwriting generation and recognition (Graves, 2012), machine translation (Bah-danau et al., 2015a), and more general computation machines (Graves et al., 2014; Weston et al., 2015).”
Grid Long Short-Term Memory (2015, methodology): “We begin by describing the standard LSTM recurrent neural network and the derived Stacked and Multidimensional LSTM networks; some aspects of the networks motivate the Grid LSTM.”
LSTM: A Search Space Odyssey (2015, neutral): “results [20], [36].”
Learning to Transduce with Unbounded Memory (2015, background): “While Deep RNNs based on long short-term memory (LSTM) cells [1, 5] can learn some transductions when tested on inputs of the same length as seen in training, they fail to consistently generalise to longer strings.”
Colorless Green Recurrent Networks Dream Hierarchically (2018, background): “More recently, RNNs have achieved impressive results in large-scale tasks such as language modeling for speech recognition and machine translation, and are by now standard tools for sequential natural language tasks (e.g., Mikolov et al., 2010; Graves, 2012; Wu et al., 2016).”
Professor Forcing: A New Algorithm for Training Recurrent Networks (2016, background): “Recurrent neural networks (RNNs) have become to be the generative models of choice for sequential data (Graves, 2012) with impressive results in language modeling (Mikolov, 2010; Mikolov and Zweig, 2012), speech recognition (Bahdanau et al., 2015; Chorowski et al., 2015), Machine Translation (Cho…”
Learning Natural Language Inference with LSTM (2015, methodology): “• mLSTM with bi-LSTM sentence modeling (d = 150): This is the same as the model above except that when we derive the hidden states hsj and h t k of the two sentences, we use bi-LSTMs (Graves, 2012) instead of LSTMs.”
A Clockwork RNN (2014, methodology): “This input format is referred to as raw in Graves (2008). All stroke sequences were sub-sampled by a factor of 3 in order to reduce the amount of data.”
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018, background): “Recurrent networks are dedicated sequence models that maintain a vector of hidden activations that are propagated through time (Elman, 1990; Werbos, 1990; Graves, 2012).”
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling (2017, methodology): “However, the right context is also important, so Bidirectional LSTMs (Graves, 2008) use two LSTMs: one for the forward pass, and another for the backward pass, LST M F and LST M B , respectively.”
Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks (2016, methodology): “The proposed network is constructed by inheriting many insights from recent successful networks (Graves 2012; Krizhevsky, Sutskever, and Hinton 2012; Szegedy et al. 2015; Du, Wang, and Wang 2015) and is designed to robustly model complex relationships among different joints.”
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, background): “The objective function of our STLSTM network is as follows: L 1⁄4PJj1⁄41PTt1⁄41 lðŷj;t; yÞ, where lðŷj;t; yÞ is the negative log-likelihood loss [70] that measures the difference between the prediction result ŷj;t at step ðj; tÞ and the true label y.”
Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, methodology): “This learning pipeline typically combines a deep convolutional neural network (CNN) [25] as visual feature extractor and an RNN [17] to model and recognize temporal dynamics of sequential data.”