Framewise phoneme classification with bidirectional LSTM and other neural network architectures

By Alex Graves, J. Urgen Schmidhuber. 2005

5,072 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Model Architectures, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Signal Processing

Citations in the corpus

Generating Sequences With Recurrent Neural Networks (2013, methodology): “To prevent this, all the experiments in this paper clipped the derivative of the loss with respect to the network inputs to the LSTM layers (before the sigmoid and tanh functions are applied) to lie within a predeﬁned range 1 .”
Semi-Supervised Sequence Modeling with Cross-View Training (2018, methodology): “The encoder applies a twolayer bidirectional LSTM (Graves and Schmidhuber, 2005) to these representations.”
Learned in Translation: Contextualized Word Vectors (2017, methodology): “We feed GloVe( w x ) to a standard, two-layer, bidirectional, long short-term memory network 1 [Graves and Schmidhuber, 2005] that we refer to as an MT-LSTM to indicate that it is this same two-layer BiLSTM that we later transfer as a pretrained encoder.”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “Meta-learning approaches include t as additional input [Schmidhuber, 1987, Thrun and Pratt, 1998, Thrun, 1998, Vilalta and Drissi, 2002].”
Adversarial Training Methods for Semi-Supervised Text Classification (2016, methodology): “For constructing the bidirectional LSTM model for text classiﬁcation, we add an additional LSTM on the reversed sequence to the unidirectional LSTM model described in Figure 1.”
Towards Automatic Learning of Procedures From Web Instructional Videos (2017, methodology): “We then forward the ResNet features through a bi-directional long short-term memory ( Bi-LSTM ) (Graves and Schmidhuber 2005) as context encoding.”
Universal Adversarial Triggers for Attacking and Analyzing NLP (2019, methodology): “We consider Bi-LSTM models (Graves and Schmidhuber, 2005) using word2vec (Mikolov et al.”
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data (2019, methodology): “For a Bidirectional LSTM (Graves & Schmidhuber, 2005) trained on IMDb reviews, classiﬁcation accuracy goes down from 79 .”
Sequence Transduction with Recurrent Neural Networks (2012, neutral): “…H ( W i ←− h it +W←−h←−h ←− h t+1 + b←−h ) (9) then iterating the forward and output layers from t = 1 to T : −→ h t = H ( W i −→ h it +W−→h−→h −→ h t−1 + b−→h ) (10) ot = W−→h o −→ h t +W←−h o ←− h t + bo (11) For a bidirectional LSTM network (Graves & Schmidhuber, 2005), H is implemented by Eqs.”
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (2018, background): “2): (1) Cascaded Encoder : The cascaded encoder aims at combining the representational power of RNNs and self-attention.”
Neural Approaches to Conversational AI (2018, positive): “As an example, this section describes the use of bidirectional LSTM (Graves and Schmidhuber, 2005), or bLSTM in short, in NLU tasks, following Hakkani-Tür et al. (2016) who also discussed other models for the same tasks.”
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks (2016, methodology): “The first layer of the model is a bi-directional LSTM (Graves and Schmidhuber, 2005; Hochreiter and Schmidhuber, 1997) whose hidden states are used to predict POS tags.”
Reasoning about Entailment with Neural Attention (2015, methodology): “Inspired by bidirectional LSTMs that read a sequence and its reverse for improved encoding (Graves and Schmidhuber, 2005), we introduce two-way attention for RTE.”