Learning Structured Weight Uncertainty in Bayesian Neural Networks

By Changyou Chen, Lawrence Carin. 2017

126 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Sorting out Lipschitz function approximation (2018, methodology): “Other researchers [2, 53, 46] have explicitly parameterized square orthogonal weight matrices using, for example, Householder transformations [25].”
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (2018, methodology): “The goal of this approach is similar to other approaches that employ structured approximations (Ritter et al., 2018; Louizos & Welling, 2016; Sun et al., 2017).”
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts (2018, methodology): “Since we focus on DNN models, instead of the linear functions used in [24], we set the regression model as a combination of sinusoidal functions as used in [33].”
Noisy Natural Gradient as Variational Inference (2017, methodology): “The results for PBP MV (Sun et al., 2017) and VMG (Louizos & Welling, 2016) can be found in supplement.”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “Accordingly, learning of different BNNs [518], [156] can be optimized using various Assumed Density Filtering (ADF) techniques [519] and apply them for different machine and deep learning tasks (e.”
A Scalable Laplace Approximation for Neural Networks (2018, methodology): “The posterior that we obtain is reminiscent of (Louizos & Welling, 2016) and (Sun et al., 2017), who optimise the parameters of a matrix normal distribution as their weights, which requires a modification of the training procedure.”
Action and Perception as Divergence Minimization (2020, background): “…Doshi-Velez, 2017), data priors (Louizos and Welling, 2016; Hafner et al., 2019b; Sun et al., 2019), ﬂexible posteriors (Louizos and Welling, 2016; Sun et al., 2017; Louizos and Welling, 2017; Zhang et al., 2018; Chang et al., 2019), low rank posteriors (Izmailov et al., 2018; Dusenberry et al.,…”
Fixing Variational Bayes: Deterministic Variational Inference for Bayesian Neural Networks (2018, methodology): “, 2016) and richer posterior families such as the matrix variate Normal posteriors (Sun et al., 2017).”
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks (2019, methodology): “al. [4] produced their analysis, while less work has gone into building on the methodology of the more computationally parsimonious Bayesian deep learning approaches like deep ensembles or MC dropout [32, 44, 45, 38]. It seems likely that this is partly shaped by the fact that UCI is the predominant”
The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks (2020, unknown): “We have by definition that W ∈ Rr×c ∼ MN (M, Q, P) if and only if vec(W) ∼ N (vec(M), P ⊗ Q), where vec(·) stacks the columns of a matrix and ⊗ is the Kronecker product The MN has already been used for variational inference by Louizos & Welling (2016) and Sun et al. (2017).”
Learning Neural Networks with Adaptive Regularization (2019, unknown): “[40] seek to learn Bayesian neural nets where they approximate the intractable posterior distribution using matrix-variate Gaussian distribution.”
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning (2019, methodology): “Sun et al. [2017] have the same complexity as Louizos and Welling [2016], depending on similar approximations and consider a maximum input dimension of only 16—over 16,000 times smaller than the input dimension of the task we address in §4.1.”
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient (2018, methodology): “Other types of approximation methods include Bayesian dropout [10] and methods that use matrix-variate Gaussians [21, 33].”
Functional Variational Bayesian Neural Networks (2019, background): “[31] Shengyang Sun, Changyou Chen, and Lawrence Carin.”
Implicit Variational Inference with Kernel Density Ratio Fitting (2017, methodology): “D Gaussian mixture distribution with variational inference.”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, background): “More recently, [3] introduce a mixture-of-Gaussians prior on the weights, with one mixture tightly concentrated around zero, thus approximating a spike and slab prior over weights.”
Eigenvalue Corrected Noisy Natural Gradient (2018, unknown): “In comparison to existing methods that ﬁt MVG posteriors [Sun et al., 2017, Louizos and Welling, 2016], noisy K-FAC does not assume additional approximations.”
A survey of uncertainty in deep neural networks (2021, background): “1[20] 3[30] 4[149] 5[75] 6[213] 7[129] 8[63] 9[80] 10[150] 11[143] 12[84] 13[235] 14[145] 15[135] 16[146] 17[272]”
Kernel Implicit Variational Inference (2017, methodology): “However, previous methods used variational posteriors with a limited capacity for each q φ l ( W l ) , including factorized Gaussian (Hernandez-Lobato & Adams, 2015), matrix variate Gaussian (Louizos & Welling, 2016; Sun et al., 2017) and normalizing ﬂows (Louizos & Welling, 2017).”
Deterministic Variational Inference for Robust Bayesian Neural Networks (2018, background): “, 2016) and richer posterior families such as the matrix variate Normal posteriors (Sun et al., 2017).”