Bayesian Back-Propagation

By Wray L. Buntine, Andreas S. Weigend. 1991

Subfields: Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, methodology): “ng Applying Bayesian methods to neural networks has been studied in the past with various approximation methods for the intractable true posterior probability distribution p(wjD). Buntine and Weigend [5] started to propose various maximum-a-posteriori (MAP) schemes for neural networks. They were als”
Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “The idea of applying Bayesian methods to neural networks has a rich history in machine learning (MacKay, 1992; Hinton & van Camp, 1993; Buntine & Weigend, 1991; Neal, 1995; De Freitas, 2003) and Mnih & Gregor (2014), where a neural network is used in a variational approximation to the posterior…”
Deep learning in neural networks: An overview (2014, background): “In the NN case, the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view (e.g., MacKay, 1992; Buntine and Weigend, 1991; De Freitas, 2003), and to high generalization performance (e.”
Weight Uncertainty in Neural Networks (2015, methodology): “In this work, we introduce an efﬁcient, principled algorithm for regularisation built upon Bayesian inference on the weights of the network (MacKay, 1992; Buntine and Weigend, 1991; MacKay, 1995).”
Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, background): “Priors for neural networks Classic work has investigated entropic priors (Buntine and Weigend, 1991) and hierarchical priors (MacKay, 1992b; Neal, 2012; Lampinen and Vehtari, 2001).”
Bayesian approach for neural networks--review and case studies (2001, methodology): “For neural networks the Bayesian approach was pioneered in (Buntine and Weigend, 1991; MacKay, 1992; Neal, 1992) and reviewed in (MacKay, 1995; Neal, 1996; Bishop, 1995).”
Recent advances in ligand-based drug design: relevance and utility of the conformationally sampled pharmacophore approach. (2011, methodology): “A variation of neural network is the Bayesian regularized neural network method which can be used to model nonlinear QSAR data [47,57-60].”
When Networks Disagree: Ensemble Methods for Hybrid Neural Networks (1992, methodology): “…and regression problems (Cooper, 1991; Reilly et al., 1988; Reilly et al., 1987; Scooeld et al., 1991; Baxt, 1992; Bridle and Cox, 1991; Buntine and Weigend, 1992; Hansen and Salamon, 1990; Intrator et al., 1992; Jacobs et al., 1991; Lincoln and Skrzypek, 1990; Neal, 1992a; Neal,…”
Extremely randomized trees (2006, background): “For example, Bayesian averaging (Buntine and Weigend, 1991; Buntine, 1992) is essentially a variance reduction technique, whereas Stacking and Boosting (Wolpert, 1992; Freund and Schapire, 1995) essentially target a reduction of bias (actually, Boosting also reduces variance; Bauer and Kohavi, 1999)”
Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors (2018, background): “Early work can be traced back to (Buntine & Weigend, 1991; MacKay, 1992; Neal, 1993).”
Bayesian Learning via Stochastic Dynamics (1992, background): “A Bayesian approach to handling this is to assign 0" a vague prior distribution and then ·.ntcgrating it away, giving the following probability for the training set (see (Buntine and Weigend, 1991) or (Neal, 1992) for details):”
A comparative predictive analysis of neural networks (NNs), nonlinear regression and classification and regression tree (CART) models (2005, methodology): “Statistical methods such as regression analysis, multivariate analysis, Bayesian theory, pattern recognition and least square approximation models have been applied to a wide range of decisions in many disciplines (Buntine & Weigend, 1991).”
Bayesian Backpropagation Over I-O Functions Rather Than Weights (1993, methodology): “) to low order and then approximate PwIT(w I t) as a sum of Gaussians (Buntine and Weigend, 1991).”
Neural networks for pattern recognition (1995, methodology): “An alternative approach is to perform the integrations over a and 0 analytically (Buntine and Weigend, 1991; Wolpert, 1993; MacKay, 1994b; Williams, 1995).”
Bayesian Recurrent Neural Networks (2017, background): “Buntine & Weigend (1991) also suggest using second order derivatives in the prior to encourage smoothness of the resulting network.”