Training recurrent networks using the extended Kalman filter

By Ronald J. Williams. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “…extended Kalman filter (Feldkamp, Prokhorov, Eagen, & Yuan, 1998; Feldkamp, Prokhorov, & Feldkamp, 2003; Haykin, 2001; Kalman, 1960; Puskorius & Feldkamp, 1994; Williams, 1992b) and an LSTM RNN (Pérez-Ortiz et al., 2003) learned to deal correctly with values of n up to 10 million and more.”
Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability (2001, background): “where w(k) is the N(N + p + 1) × 1 vector obtained by rearranging the weight matrix W (k), x(k) is an M × 1 observation vector, q(k) ∼ N (0, Q) is a vector of white Gaussian noise (WGN), as well as v(k) ∼ N (0, C) (Williams 1992).”
Mercury: Metro density prediction with recurrent neural network on streaming CDR data (2016, background): “Recurrent neural network is proposed in early 90s last century [18].”
A signal processing framework based on dynamic neural networks with application to problems in adaptation, filtering, and classification (1998, background): “On the other hand, Matthews [20] and Williams [ 21 ] have described parallel EKF formulations for recurrent networks in which both network states and weights are updated during training.”
A Clockwork RNN (2014, methodology): “Training RNNs with Kalman ﬁlters (Williams, 1992) has shown advantages in bridging long time lags as well, although this approach is computationally unfeasible for larger networks.”
Recent Advances in Recurrent Neural Networks (2017, methodology): “Similar to SGD, it can train a RNN with incoming input data in an online manner [33].”