Bayesian Methods for Adaptive Models

By John Scott Bridle, P. Cheeseman, S. Fels, Stephen F. Gull, Andreas V. M. Herz, John J. Hopfield, D. Kerns, Alan P. Knutsen, David Koerner, Mike Lewicki and 12 more. 2011

403 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Hence, different degrees of approximations are made to reduce the computational overhead [40], [41], [42], [43], including about the prior that is specified for the parameters of the neural network.”
Uncertainty-guided Continual Learning with Bayesian Neural Networks (2019, methodology): “Bayesian approaches: Using Bayesian approach in learning neural networks has been studied for few decades (MacKay, 1992b;a).”
Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, background): “The largest body of research on obtaining uncertainty in deep learning are Bayesian neural networks (MacKay, 1992; Neal, 2012).”
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks (2015, methodology): “We further compare with the Laplace approximation (LA) of MacKay (1992b) using the neural network toolbox from Matlab with tanh nonlinearities.”
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift (2019, background): “Popular approximate Bayesian approaches include Laplace approximation (MacKay, 1992), variational inference (Graves, 2011; Blundell et al., 2015), dropout-based variational inference (Gal & Ghahramani, 2016; Kingma et al., 2015), expectation propagation Hern´andez-Lobato & Adams (2015) and…”
Practical Deep Learning with Bayesian Principles (2019, methodology): “, by using MCMC methods [41], Laplace’s method [35], and variational inference (VI) [18, 2, 49, 1].”
Deep Ensembles: A Loss Landscape Perspective (2019, methodology): “A variety of approximations have been developed for Bayesian neural networks, including Laplace approximation [MacKay, 1992], Markov chain Monte Carlo methods [Neal, 1996, Welling and Teh, 2011, Springenberg et al.”
Bayesian Deep Learning and a Probabilistic Perspective of Generalization (2020, methodology): “A similar schematic to Figure 2(a) was used by MacKay [26] to understand an Occam’s razor effect in using the evidence for model selection; we believe it can also be used to reason about model construction and generalization.”
Deep and Hierarchical Implicit Models (2017, background): “(Surprisingly, in Section 3.5 we outline how the KL is the only possible objective among a broad class.)”
Trusted Multi-View Classification With Dynamic Evidential Fusion (2022, neutral): “distribution over the weights [23], [24], [25].”
Automatic model construction with Gaussian processes (2014, background): “The crucial property of GPs that allows us to automatically construct models is that we can compute the marginal likelihood of a dataset given a particular model, also known as the evidence (MacKay, 1992).”
$L1$ -Norm Batch Normalization for Efficient Training of Deep Neural Networks (2018, methodology): “In addition, the inherent randomization incurred by the minibatch statistics serves as a regularizer and approximates inference in Bayesian models [9]–[11], making BN a better alternative to dropout [12].”
Hierarchical Implicit Models and Likelihood-Free Variational Inference (2017, methodology): “Bayesian GANs enable modeling of parameter uncertainty and are inspired by Bayesian neural networks, which have been shown to improve the uncertainty and data efficiency of standard neural networks [31, 37].”
Planning to Explore via Self-Supervised World Models (2020, methodology): “This subsection derives our method from the amount of information gained by interacting with the environment, which has its roots in optimal Bayesian experiment design (Lindley, 1956; MacKay, 1992).”
Meta-Learning with Shared Amortized Variational Inference (2020, neutral): “It is natural to explicitly model this variance by treating the parameters as latent variables in a Bayesian framework (Garnelo et al., 2018; Gordon et al., 2019; Grant et al., 2018; Kim et al., 2019; MacKay, 1991; Neal, 1995).”
Improving PILCO with Bayesian Neural Network Dynamics Models (2016, methodology): “This involves sampling a set of particles from the input distribution (step 2 in Algorithm 2), passing these particles through the BNN dynamics model (and sampling the uncertain output, step 8 in Algorithm 2), which yields an output distribution of particles.”
Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty (2021, background): “Bayesian Models Neal [2012], MacKay [1992] provide a principled way of measuring uncertainty.”