Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets
By Juan Antonio Pérez-Ortiz, Felix Alexander Gers, Douglas Eck, J. Urgen Schmidhuber. 2003
109 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- J. Urgen Schmidhuber
- Yoshua Bengio
- Sergey Levine
- Faustino J. Gomez
- Quoc V. Le
- Kyunghyun Cho
- Mohit Bansal
- Jan Koutník
In Computer Vision and Pattern Recognition
- J. Urgen Schmidhuber
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Andrew Zisserman
- Jitendra Malik
Citations in the corpus
- On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “2) will develop high-level, abstract, spatio-temporal feature detectors that may be active for thousands of time steps, as long as those memories are useful to predict (and thus compress) future observations [62, 61, 189, 79].”
- A Clockwork RNN (2014, methodology): “Training RNNs with Kalman filters (Pérez-Ortiz et al., 2003; Williams, 1992) has shown advantages in bridging long time lags as well, although this approach is computationally unfeasible for larger networks.”
- Framewise phoneme classification with bidirectional LSTM networks (2005, unknown): “In the past, training with decoupled Kalman filters [16], has given improved results for several tasks.”
- A First Look at Music Composition using LSTM Recurrent Neural Networks (2002, methodology): “LSTM performance is improved in online learning situations by using a Kalman filter to control weight updates (Pérez-Ortiz et al., 2002).”
- Evolino for recurrent support vector machines (2005, methodology): “[21, 29, 30, 31, 32, 33, 34].”
- Recent Advances in Recurrent Neural Networks (2017, methodology): “However, computation requires just the results from the previous step and there is no need to store results beyond that step [22].”
- Training Recurrent Networks by Evolino (2007, background): “Previous work on LSTM has focused on gradient-based G-LSTM [1–3, 8, 11, 29, 38].”