Laplace Propagation

By Alex Smola, S. V. N. Vishwanathan, Eleazar Eskin. 2003

74 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “the Fisher information matrix computed from the previous task. Since the diagonal Fisher information approximates the average Hessian of the likelihoods, EWC is closely related to Laplace propagation [17, 43], where Laplace’s approximation is applied after each task to compute the regularizers. Besi”
Overcoming catastrophic forgetting in neural networks (2016, methodology): “Note that this approach is similar to expectation propagation where each subtask is seen as a factor of the posterior [Eskin et al., 2004].”
Variational Continual Learning (2017, methodology): “• Laplace Propagation (Smola et al., 2004): applying Laplace approximation at each step leads to a recursion for Σ − 1 t , which is initialized using the covariance of the Gaussian prior, θ = θ t and λ t = 1.”
Generalized Variational Continual Learning (2020, background): “This recursion of Σ − 1 T gives Now, optimizing the β -ELBO for µ T (ignoring terms that do not depend on µ T ): Which is the exact optimization problem for Laplace Propagation (Smola et al., 2003).”
Bayesian Structure Adaptation for Continual Learning (2019, methodology): “Notable methods based on this strategy include EwC (Kirkpatrick et al., 2017), SI (Zenke et al., 2017), Laplace approximation (Smola et al., 2003), etc. Superceding these methods is the Bayesian approach, a natural remedy of catastrophic forgetting in that, for any task, the posterior of the model…”
Functional Regularisation for Continual Learning using Gaussian Processes (2019, background): “and standard deviation over 10 random repetitions. Where applicable, we also report the number of inducing points/replay buffer size per task in parentheses. Algorithm Permuted-MNIST Split-MNIST DLP (Eskin et al., 2004) 82% 61.2% EWC (Kirkpatrick et al., 2017) 84% 63.1% SI (Zenke et al., 2017) 86% 9”
Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting (2018, methodology): “Our method is closely related to Bayesian online learning [31] and to Laplace propagation [4].”
Lifelong Generative Modeling (2017, methodology): “2) and provide comparisons against baselines [91, 136, 65, 29, 63] using a set of standard metrics (Section 6.”
Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data (2014, methodology): “This corresponds to the Laplace approximation, and the message passing algorithm corresponds to Laplace propagation (Smola et al., 2004).”
Variational Beam Search for Novelty Detection (2021, unknown): “In addition to Variational Continual Learning (VCL, see Section 2), we also compared against Laplace Propagation (LP) (Smola et al., 2003).”
Variational Federated Multi-Task Learning (2019, methodology): “These methods have been first introduced as heuristics, but have been found to be applications of wellknown inference algorithms like Laplace Propagation [41] and Streaming Variational Bayes [7], which led to further generalizations [26, 16].”
Variational inference in nonconjugate models (2012, unknown): “Smola et al. (2003) used them to approximate the difﬁcult-to-compute moments in expectation propagation (Minka, 2001).”
Expectation Propagation as a Way of Life (2020, unknown): “This corresponds to the Laplace approximation (Smola et al., 2004).”