A Practical Bayesian Framework for Backprop Networks

By David John Cameron MacKay. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, methodology): “Sevreal approaches has been proposed in the past for learning in Bayesian Networks: Laplace approximation [43], MC Dropout [13],and Variational Inference [26] [19] [4].”
Deep learning in neural networks: An overview (2014, methodology): “Often weight priors are implicit in additional penalty term s (MacKay, 1992) or in methods based on validation sets (Mosteller and Tukey, 1968; Stone, 1974; Eubank, 1988; Hast ie and Tibshirani, 1990; Craven and Wahba, 1979; Golub et al.”
Natural-Parameter Networks: A Class of Probabilistic Neural Networks (2016, methodology): “Early BNN works include methods based on Laplace approximation [16], variational inference (VI) [11], and Monte Carlo sampling [18], but they have not been widely adopted due to their lack of scalability.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “2 M ’s Training To decrease bitsM + bitsH , we add a regularizing term to E, to punish excessive complexity [4, 5, 91, 294, 155, 135, 97, 170, 169, 104, 290, 7, 290, 87, 286, 319, 100, 102].”
A Survey on Bayesian Deep Learning (2016, methodology): “The study of BNN dates back to 1990s with notable works from [42, 72, 80].”
Flat Minima (1997, methodology): “B(w; X 0 )), but it is not obvious how: the \quick and dirty version" (MacKay, 1992a) cannot deal with the unknown constant in B(w; X 0 ).”
Towards Bayesian Deep Learning: A Survey (2016, methodology): “2, if we assume that both the clean input Xc and the corrupted input X0 are observed, similar to [2], [3], [9], [37], we can define the following generative process of generalized Bayesian SDAE:”
Gaussian Processes for Regression and Optimisation (2007, methodology): “For good introductions to Gaussian processes for regression refer to the 1997 thesis of Gibbs [17], the Gaussian processes chapter in MacKay's book [34], and the recent book by Williams and Rasmussen [61].”
A Simple Weight Decay Can Improve Generalization (1991, background): “Weight decay as a special kind of regularization is also discussed in [8] [9] .”
Bayesian Methods for Adaptive Models (2011, methodology): “Multiple maxima which arise in more complex models complicate the analysis, but Bayesian methods can still successfully be applied [31, 50, 55].”
Fast Exact Multiplication by the Hessian (1994, methodology): “, 1991; Pearlmutter, 1992); in some techniques for predicting generalization rates in neural networks (MacKay, 1991; Moody, 1992); in techniques for enhancing generalization by weight elimination (le Cun et al.”
Exact Calculation of the Product of the Hessian Matrix of Feed-Forward Network Error Functions and a Vector in 0(N) Time (1993, methodology): “…is needed in several learning algorithms, e.g., in some conjugate gradient algorithms This work was done while the author was visiting School of Computer Science, Carnegie Mellon University, Pittsburgh, PA. [M ller 93a], and in recent network pruning techniques [MacKay 91], [Hassibi 92].”
Bayesian Regularization and Pruning Using a Laplace Prior (1995, methodology): “The evidence framework (MacKay 1992; Thodberg 1993) proposes to set the regularizing parameters (Y and i;l by maximizing”
Large-scale clinical interpretation of genetic variants using evolutionary data and deep learning (2020, background): “While a standard VAE will then decode that mutant to an arbitrary position, the output from a Bayesian VAE will average over decodings (the decoder being a Bayesian Neural Network), which will dampen the corresponding probability estimates over amino acid positions [22].”