Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks

By G. Puskorius, Lee A. Feldkamp. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “A combination of a decoupled extended Kalman filter (Kalman, 1960; Williams, 1992b; Puskorius and Feldkamp, 1994; Feldkamp et al., 1998; Haykin, 2001; Feldkamp et al., 2003) and an LSTM RNN (Pérez-Ortiz et al.”
On-line learning algorithms for locally recurrent neural networks (1999, methodology): “Of course the gradient calculation techniques developed in this paper can be implemented in second-order methods such as conjugate gradient or Kalman filter based algorithms [29], but this is beyond the aim of the paper.”
Sequence Learning (2001, methodology): “Among these the most popular algorithms are Back-propagation Through Time (BPTT) [44], Real Time Recurrent Learning (RTRL) [75] developed for on-line training, Kalman (Extended) Filter (EKF) [74] [49], and Temporal Difference [62] [63].”
Recurrent convolutional neural network for object recognition (2015, background): “Recurrent neural network (RNN) has a long history in the artificial neural network community [4, 21, 11, 37, 10, 24], but most successful applications refer to the modeling of sequential data such as handwriting recognition [18] and speech recognition[19].”
Long-term wind speed and power forecasting using local recurrent neural network models (2006, methodology): “Furthermore, the recursion (19)–(23) has strong similarities to the extended Kalman filter (EKF) algorithm used in [23].”
Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets (2003, methodology): “The DEKF (Puskorius and Feldkamp, 1994; Haykin, 1999) overcomes this limitation.”
Recent Advances in Recurrent Neural Networks (2017, methodology): “A more efficient and effective version of EKF is the decoupled EKF (DEKF) method, which ignores the interdependencies of mutually exclusive groups of weights [32].”
An overview of reservoir computing: theory, applications and implementations (2007, background): “Nonetheless, the application of these rules to real-world problems is not always feasible (due to the high computational training costs and slow convergence) [14] and while it is possible to achieve state-of-the-art performance, this is only reserved for experts in the field [38].”
Long Short-Term Memory (1997, methodology): “Puskorius and Feldkamp (1994) use Kalman filter techniques to improve recurrent net performance.”
A unified theory of adaptive stochastic gradient descent as Bayesian filtering (2018, methodology): “Here we provide such a theory by returning to very early work that used Bayesian (Kalman) ﬁltering to optimize the parameters of neural networks (Puskorius & Feldkamp, 1991; Sha et al., 1992; Puskorius & Feldkamp, 1994; 2001; Feldkamp et al., 2003).”