Ensemble learning in Bayesian neural networks

By David Barber, Charles M. Bishop. 1998

Subfields: Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “…published in 1993) made important connections to the expectation maximization (EM) algorithm (Dempster, Laird, and Rubin 1977), which then led to a variety of variational inference algorithms for other types of models (Waterhouse, MacKay, and Robinson 1996; MacKay 1997; Barber and Bishop 1998).”
Weight Agnostic Neural Networks (2019, background): “The weight parameters of a BNN [3, 4, 26, 43, 68, 78] are not ﬁxed values, but sampled from a distribution.”
A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, neutral): “Many have followed this approach in the past for standard NN models [26], [3], [19], [4].”
Variational Continual Learning (2017, background): “…including extended Kalman ﬁltering (Singhal and Wu, 1989), Laplace approximation (MacKay, 1992), variational inference (Hinton and Van Camp, 1993; Barber and Bishop, 1998; Graves, 2011; Blundell et al., 2015; Gal and Ghahramani, 2016), sequential Monte Carlo (de Freitas et al., 2000), EP…”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “Existing approaches to approximate this posterior are through variational inference (as was done in Hinton & Van Camp (1993); Barber & Bishop (1998); Graves (2011); Blundell et al. (2015)).”
Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference (2015, methodology): “Many have followed this approach in the past for standard MLP models [6, 7, 8, 9].”
Practical Deep Learning with Bayesian Principles (2019, methodology): “, by using MCMC methods [41], Laplace’s method [35], and variational inference (VI) [18, 2, 49, 1].”
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (2018, neutral): “M L ] 2 A ug 2 01 8 rithms such as MCMC methods (Neal, 1995), Laplace’s method (Denker & Lecun, 1991), and variational inference (Hinton & Van Camp, 1993; Barber & Bishop, 1998).”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, background): “…and Wu, 1989], Laplace’s approximation [MacKay, 2003], Hamiltonian Monte Carlo [Neal, 1993, 2012], variational inference [Hinton and Van Camp, 1993, Barber and Bishop, 1998, Graves, 2011, Blundell et al., 2015, Gal and Ghahramani, 2016], sequential Monte Carlo [de Freitas et al., 2000],…”
Pattern Recognition and Machine Learning (2006, unknown): “In this sense, it is analogous to the choice of a restrictive form f covariance matrix (for example, a diagonal matrix) in a multivariate Gaussian distribution.”
Robustly representing inferential uncertainty in deep neural networks through sampling (2016, methodology): “…minimization can be reformulated in terms of the prior on the weights p ( W ) and the data likelihood p ( D Train | W ) (Hinton & Van Camp, 1993; Barber & Bishop, 1998; Graves, 2011; Blundell et al., 2015): Using Monte Carlo (MC) methods to estimate E q V ( w ) [ p ( D train | W )] , using…”
Variational Implicit Processes (2018, background): “Although decent progress has been made [4,9,20,31,36,41,54,69], uncertainty in deep learning still remains an open challenge.”
Deterministic Variational Inference for Robust Bayesian Neural Networks (2018, methodology): “Two recent works derived deterministic moment approximations for deep neural networks: Bibi et al. (2018), using Price’s theorem, derived exact first and second moment expressions for ReLU activations but limit themselves to the case of zero-mean Gaussian activations.”
Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks (2019, methodology): “In earlier works, Bayesian DNNs focused on the use of Monte Carlo or ensemble based methodologies [50, 51, 9].”
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (2015, methodology): “Recent advances in variational inference introduced new techniques into the ﬁeld such as sampling-based variational inference and stochastic variational inference (Blei et al., 2012; Kingma & Welling, 2013; Rezende et al., 2014; Titsias & L´azaro-Gredilla, 2014; Hoffman et al., 2013).”
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision (2019, background): “The second paradigm is that of Variational Inference (VI) [22, 2, 16, 4].”
An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models (2002, methodology): “Ensemble learning (Barber and Bishop, 1998; Lappalainen and Miskin, 2000), also called variational Bayes, is a method developed recently (Hinton and van Camp, 1993; MacKay, 1995) for approximating the posterior density (2).”
Uncertainty in Deep Learning (2016, methodology): “traced back as far as [Hinton and Van Camp, 1993] and Barber and [Barber and Bishop, 1998]).”
Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks (2018, methodology): “techniques for obtaining Bayesian statistics include the placement of distributions over network weights and sampling with Monte Carlo methods to approximate statistical bounds [32, 33] as well as ensemble methods [34, 35].”