Bayesian Learning via Stochastic Dynamics

By Radford M. Neal. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Stochastic Gradient Hamiltonian Monte Carlo (2014, background): “However, as we have demonstrated, the idea is crucial in our stochastic gradient scenario in order to counterbalance the effect of the noisy gradients.”
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks (2017, methodology): “There exist many techniques for making neural networks more Bayesian [17, 44, 43, 21].”
Unbiased Markov chain Monte Carlo methods with couplings (2020, methodology): “Going further in the use of gradient information, the Hamiltonian or hybrid Monte Carlo algorithm (Duane et al., 1987; Neal, 1993, 2011) is a popular MCMC algorithm for large dimensional targets.”
Keeping the neural networks simple by minimizing the description length of the weights (1993, unknown): “Neal (1993) shows how the gradient information provided by backpropagation can be used to get a much more efficient method of obtaining samples from the posterior distribution.”
Bean Machine: A Declarative Probabilistic Programming Language For Efficient Programmable Inference (2020, methodology): “The second is that it allows state-of-the-art gradient-based proposers such as Hamiltonian Monte Carlo (HMC) (Duane et al., 1987; Neal, 1993), No U-Turn Sampler (Hoffman and Gelman, 2014), and Newtonian Monte Carlo (NMC) (Arora et al., 2020), to be used on models with mixed discrete and continuous…”
In Advances in Neural Information Processing Systems (2011, unknown): “For neural networks the prior over functions has a complex form which meansthat implementations must either make approximations (e.g. MacKay, 1992) or useMonte Carlo approaches to evaluating integrals (Neal, 1993).”
Deterministic Variational Inference for Robust Bayesian Neural Networks (2018, methodology): “a reasonable computation budget (Kingma et al., 2015; Molchanov et al., 2017; Miller et al., 2017; Zhu et al., 2018). Second, performance of the Bayesian approach is sensitive to the choice of prior (Neal, 1993), and although we may have a priori knowledge concerning the function represented by a ne”
HYPERPARAMETERS: OPTIMIZE, OR INTEGRATE OUT? (1996, methodology): “Recent developments (Neal 1993a; Skilling 1993) now make it possible to use gradient and curvature information so as to sample high dimensional spaces more e ectively, even for highly non-Gaussian distributions.”
Fast, frugal, and rational: How rational norms explain behavior (2003, methodology): “developed, as touched on briefly above (e.g., Chater, 1995; McClelland, 1998; Mackay, 1992; Neal, 1993; Nosofsky, 1990).”
Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights (2020, methodology): “Note that the periodic kernel allows the BNN to discover and encode periodicity in its weights, leading to long-range confident predictions compared to that of the RBF kernel and significantly better extrapolation than BNNs with independent weight priors can obtain given the amount of training data,”
Bayesian Backpropagation Over I-O Functions Rather Than Weights (1993, background): “), the correction term has no effect for some non-MAP-based ways to apply Bayesianism to neural nets, like guessing the posterior average ct» (Neal, 1993):”
On the convergence of Hamiltonian Monte Carlo (2017, background): “It has been popularized in the computational statistics community by the landmark paper by [11] to infer Bayesian neural networks.”
Riemann manifold Langevin and Hamiltonian Monte Carlo methods (2011, background): “Viewed in this form it is clear that the choice of the mass matrix M, as in the MALA, will be critical for the performance of HMC sampling, and like the MALA there is no guiding principle on how this should be chosen and tuned. The demonstrated ability of HMC sampling to overcome random walks in MCM”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, methodology): “…of neural networks, including extended Kalman filtering [Singhal and Wu, 1989], Laplace’s approximation [MacKay, 2003], Hamiltonian Monte Carlo [Neal, 1993, 2012], variational inference [Hinton and Van Camp, 1993, Barber and Bishop, 1998, Graves, 2011, Blundell et al., 2015, Gal and…”
A Scalable Laplace Approximation for Neural Networks (2018, methodology): “There has been previous interest in integrating neural networks into the Bayesian framework (MacKay, 1992; Hinton & Van Camp, 1993; Neal, 1993; Barber & Bishop, 1998), however these approaches were designed for small networks by current standards.”