Bayesian Methods for Backpropagation Networks

By David John Cameron MacKay. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, positive): “Fortunately, some classic yet powerful techniques in Bayesian learning are applicable, i.e., sparse Bayesian learning (SBL) (Tipping, 2001; Pan, 2017) and automatic relevance determination (ARD) prior (MacKay, 1996; Neal, 1995) in Bayesian neural networks.”
Calibrating Deep Convolutional Gaussian Processes (2018, background): “Despite the abundant literature on Bayesian DNN s [22, 21], far less attention has been devoted to Bayesian CNN s [9], and the calibration properties of these approaches have not been investigated.”
Bayesian Model and Dimension Reduction for Uncertainty Propagation: Applications in Random Media (2017, methodology): “he underlying physics of the problem. Several sparsity enforcing approaches were tested in this work, including the Laplacian prior (or LASSO regression [71,26]) as well as Student-ttype prior models [45,55,22]. We achieved best experimental results using a slightly modied version of the Relevance V”
Sparse Bayesian Learning and the Relevance Vector Machine (2001, methodology): “We term those training vectors associated with the remaining non-zero weights `relevance' vectors, in deference to the principle of automatic relevance determination which motivates the presented approach (MacKay, 1994; Neal, 1996).”
In Advances in Neural Information Processing Systems (2011, unknown): “…estimated from the data, using either a maximum likelihood or Bayesianapproach, and that this leads to a form of \Automatic Relevance Determination"(Mackay 1993; Neal 1995).2 PREDICTION WITH GAUSSIAN PROCESSESA stochastic process is a collection of random variables fY (x)jx 2 Xg indexed by aset X.”
In Advances in Neural Information Processing Systems (1996, unknown): “Neal added four further inputs, two of which were copies of Xl and X2 corrupted by additive Gaussian noise of standard deviation 0.02 , and two further irrelevant Gaussian-noise inputs with zero mean and unit variance.”
Gaussian Processes for Regression and Optimisation (2007, methodology): “Conversely, if rd is large, the model will be very smooth in the d axis direction, and the d components of the inputs xi and xj could be said to be largely irrelevant, as in the automatic relevance determination (ARD) method for neural networks [37, 46, 34].”
Preference learning with Gaussian processes (2005, methodology): “The Laplace approximation of S(f) refers to carrying out the Taylor expansion at the MAP point and retaining the terms up to the second order (MacKay, 1994).”
Random Feature Expansions for Deep Gaussian Processes (2016, methodology): “After explaining the random feature expansion associated with each covariance, we will generalize these results in the context of DGPs to include scaling the covariance by a factor σ(2), and scaling the features for Automatic Relevance Determination (ARD) (Mackay, 1994).”
Bayesian Classification With Gaussian Processes (1998, methodology): “This is closely related to the Automatic Relevance Determination (ARD) idea of MacKay [10] and Neal [15].”
Model‐based geostatistics (2007, unknown): “This a ords an implementation of the `automatic relevance determination' idea of MacKay (1993) and Neal (1996).”
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond (1999, methodology): “This is closely related to the Automatic Relevance Determination (ARD) idea of MacKay and Neal (MacKay, 1993; Neal 1996).”
Multi-task compressive sensing with Dirichlet process priors (2008, background): “…10-4) to make a broad prior over ak , which allows the posteriors on many of the elements of ato concentrate at very large k values, consequently the posteriors on the associated elements of .i concentrate at zero, and therefore the sparseness of .i is achieved (MacKay, 1994) (Neal, 1996).”
Information Theory, Inference, and Learning Algorithms (2004, methodology): “…but we don’t know which, we can define a new model with multiple hyperparameters that captures the idea of uncertain input variable relevance (MacKay, 1994b; Neal, 1996; MacKay, 1995b); these models then infer automatically from the data which are the relevant input variables for a…”