Long short-term memory in recurrent neural networks

By Felix Alexander Gers. 2001

274 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

SampleRNN: An Unconditional End-to-End Neural Audio Generation Model (2016, methodology): “For the case of LSTMs, the forget gate bias is initialized with a large positive value of 3, as recommended by Zaremba (2015) and Gers (2001), which has been shown to be beneficial for learning long-term dependencies.”
A Critical Review of Recurrent Neural Networks for Sequence Learning (2015, methodology): “Among the most useful resources are a recent book on supervised sequence labeling with recurrent neural networks [Graves, 2012] and an earlier doctoral thesis [Gers, 2001].”
Sequential User-based Recurrent Neural Network Recommendations (2017, background): “Both Long Short-Term Memory (LSTM) [9, 20] and Gated Recurrent Unit (GRU) [5, 6] networks have shown advantages in real-world applications.”
Sequence Transduction with Recurrent Neural Networks (2012, background): “Bidirectional RNNs are preferred because each output vector depends on the whole input sequence (rather than on the previous inputs only, as is the case with normal RNNs); however we have not tested to what extent this impacts performance.”
Joint Event Extraction via Recurrent Neural Networks (2016, background): “This problem can be alleviated by long-short term memory units (LSTM) (Hochre-iter and Schmidhuber, 1997; Gers, 2001).”
Long short-term memory neural network for traffic speed prediction using remote microwave sensor data (2015, background): “Errors are truncated when they arrive at a memory cell output, and then, they enter the memory cell’s linear CEC, where errors can flow back forever, and making error flow outside the cell tends to decay exponentially (Gers, 2001).”
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network (2018, background): “of the cell is refreshed on every step19. Essentially, the multiplicative gate units open and close access to constant error gradient ﬂow through CEC as part of the operation of the LSTM cell20 [14], [7]. 18Note the notation change: in the rest of the document, the symbol, ~u[n], has the meaning of ”
Context-Dependent Sentiment Analysis in User-Generated Videos (2017, methodology): “To capture this ﬂow of informational triggers across utterances, we use a LSTM-based recurrent neural network (RNN) scheme (Gers, 2001).”
Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory (2015, methodology): “An effective solution for these problems is the Long Short-Term Memory (LSTM) architecture (Hochreiter and Schmidhuber, 1997; Gers, 2001).”
Supervised Sequence Labelling with Recurrent Neural Networks (2012, background): “, 2002) connecting the gates to the memory cell were added later to give extended LSTM (Gers, 2001).”