Sequential Monte Carlo Methods to Train Neural Network Models

By J. F. G. D. Freitas, Mahesan Niranjan, Andrew Howard Gee, Arnaud Doucet. 2000

234 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Variational Continual Learning (2017, background): “…1992), variational inference (Hinton and Van Camp, 1993; Barber and Bishop, 1998; Graves, 2011; Blundell et al., 2015; Gal and Ghahramani, 2016), sequential Monte Carlo (de Freitas et al., 2000), EP (Hern´andez-Lobato and Adams, 2015), and approximate Power EP (Hern´andez-Lobato et al., 2016).”
Deep Rewiring: Training very sparse deep networks (2017, methodology): “Second, in DEEP R, gradient descent is combined with a random walk in parameter space (de Freitas et al., 2000; Welling & Teh, 2011).”
Adaptive prognosis of lithium-ion batteries based on the combination of particle filters and radial basis function neural networks (2017, methodology): “Thus, according to [19] and to many other works of literature (e.g. [15]) on fixed parameter estimation by particle filtering, the evolution of the network weights during their updating process is modeled as a discrete Markov chain driven by a process noise 𝝎 𝑘 which allows to properly “explore”…”
An Introduction to MCMC for Machine Learning (2004, methodology): “For instance, one can adopt suboptimal filters and other approximation methods that make use of the information available at time t to generate the proposal distribution (Doucet, Godsill, & Andrieu, 2000; de Freitas et al., 2000; Pitt & Shephard, 1999; van der Merwe et al., 2000).”
The Unscented Particle Filter (2000, methodology): “For example, in (de Freitas et aL 2000), the EKF Gaussian approximation is used as the proposal distribution for a PF.”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, background): “…[Hinton and Van Camp, 1993, Barber and Bishop, 1998, Graves, 2011, Blundell et al., 2015, Gal and Ghahramani, 2016], sequential Monte Carlo [de Freitas et al., 2000], expectation propagation [Hernández-Lobato and Adams, 2015], and approximate power EP [Li et al., 2015, Hernández-Lobato et…”
Multi-modal estimation with kernel embeddings for learning motion models (2013, background): “EKF and UKF particle filters use those filters to approximate the proposal distribution, reducing sample impoverishment but introducing linearisation errors [13], [14].”
A Latent Variable Recurrent Neural Network for Discourse-Driven Language Models (2016, methodology): “Learning Online learning was performed using AdaGrad (Duchi et al., 2011) with initial learning rate λ = 0.1.”
FastSLAM 2.0: an improved particle filtering algorithm for simultaneous localization and mapping that provably converges (2003, methodology): “This extension parallels prior work by Doucet and colleagues, who proposed a similar modification for general particle filters [6] and Markov Chain Monte Carlo techniques for neural networks [4].”