Training Multilayer Perceptrons with the Extende Kalman Algorithm

By S. Singhal, Lance Wu. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Continual Learning (2017, neutral): “There is a long history of research on approximate Bayesian approaches to training neural networks, including extended Kalman ﬁltering (Singhal and Wu, 1989), Laplace approximation (MacKay, 1992), variational inference (Hinton and Van Camp, 1993; Barber and Bishop, 1998; Graves, 2011; Blundell et…”
New Insights and Perspectives on the Natural Gradient Method (2014, background): “…flavor of Tikhonov regularization, closely resembles a certain type of extended Kalman filter-based training algorithm for neural networks (Singhal and Wu, 1989; Ruck et al., 1992), where θ is treated as an evolving hidden state that is estimated by the filter (using training targets as…”
Bayesian methods for neural networks (2000, background): “One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu (Singhal and Wu, 1988).”
A Machine Learning Perspective on Predictive Coding with PAQ8 (2012, background): “One of the earliest implementations of the EKF to train multilayer perceptrons is due to Singhal and Wu [14].”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing (2019, methodology): “For example, in [54] it was shown that training an MLP using EKF requires orders of magnitude lower number of epochs than standard back-propagation.”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, methodology): “There is a long history of research on approximate Bayesian training of neural networks, including extended Kalman filtering [Singhal and Wu, 1989], Laplace’s approximation [MacKay, 2003], Hamiltonian Monte Carlo [Neal, 1993, 2012], variational inference [Hinton and Van Camp, 1993, Barber and Bishop”
Analysis and Design of Echo State Networks (2007, methodology): “Alternative second-order training methods based on extended Kalman filtering (Singhal & Wu, 1989; Puskorius & Feldkamp, 1994; Feldkamp, Prokhorov, Eagen, & Yuan, 1998) and the multistreaming training approach (Feldkamp et al., 1998) provide more reliable performance and have enabled practical…”
Sequential Monte Carlo Methods to Train Neural Network Models (2000, background): “One of the earliest implementations of EKF-trained MLPs is due to Singhal and Wu (1988).”