Generalized Beta Mixtures of Gaussians

By Artin Armagan, David B. Dunson, Merlise A. Clyde. 2011

164 citations recorded by the source corpus.

In recent years, a rich variety of shrinkage priors have been proposed that have great promise in addressing massive regression problems. In general, these new priors can be expressed as scale mixtures of normals, but have more complex forms and better properties than traditional Cauchy and double exponential priors. We first propose a new class of normal scale mixtures through a novel generalized beta distribution that encompasses many interesting priors as special cases. This encompassing framework should prove useful in comparing competing priors, considering properties and revealing close connections. We then develop a class of variational Bayes approximations through the new hierarchy presented that will scale more efficiently to the types of truly massive data sets that are now encountered routinely.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, methodology): “General variational inference algorithms have been developed for a variety of classes of models, including shrinkage models (Armagan et al., 2011; Armagan and Dunson, 2011; Neville et al., 2014), general time-series models (Roberts et al.”
Bayesian Compression for Deep Learning (2017, background): “In fact, several well known shrinkage priors can be expressed in this form by altering the shapes of the (inverse) Gamma hyperpriors [Armagan et al., 2011].”
Bayesian group factor analysis with structured sparsity (2016, methodology): “The three parameter beta (T PB) distribution for a random variable Z ∈ (0, 1) has the following density (Armagan et al., 2011): f (z; a, b, ν) = Γ(a + b) Γ(a)Γ(b) ν b z b−1 (1 − z) a−1 {1 + (ν − 1)z} −(a+b) , (10) Figure 2: Density of three parameter beta (T PB) distribution with different values…”
Simple Marginally Noninformative Prior Distributions for Covariance Matrices (2013, background): “The result for the Half-Cauchy case also appears as Proposition 1 in Armagan et al. (2011).”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “ew have large eect, without explicitly having prior probability at j = 0. Some examples include normal-gamma mixtures (Grin et al., 2010), Horseshoe (Carvalho et al., 2009, 2010), generalized Beta (Armagan et al., 2011), generalized double Pareto (Armagan et al., 2013a), Dirichlet-Laplace (Bhattacha”
Hierarchical Shrinkage Priors for Regression Models (2017, background): “, 2010), the normal-exponential-gamma (NEG) (Griffin and Brown, 2011), the generalized Beta mixtures (Armagan et al., 2011), the generalized t (Lee et al.”
Polygenic prediction via Bayesian regression and continuous shrinkage priors (2018, methodology): “…and Doucet, 2008; Grifﬁn and Brown, 2010], the normal-inverse-gaussian prior [Caron and Doucet, 2008], the generalized t (generalized double Pareto) prior [Armagan et al., 2013; Lee et al., 2012], and the normal-exponential-gamma prior [Armagan et al., 2011; Grifﬁn and Brown, 2011], among others.”
Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies (2014, unknown): “Results 1a and 1b follow from results on the Horseshoe distribution given in Carvalho, Polson and Scott [9] as well as Proposition 1 of Armagan, Dunson and Clyde [4] or Result 5 of Wand et al. [34].”
Sparse Multi-Output Gaussian Processes for Medical Time Series Prediction (2017, methodology): “Following Proposition 1 in Armagan et al. (2011), for a random variable x drawn from a normal distribution with two-layered gamma priors on variance x ∼ N (0, ψ1), ψ1 ∼ G(α, δ), δ ∼ G(β, ν), (28) is equivalent to the hierararchy x ∼ N (0, 1/ρ− 1), ρ ∼ T PB(α, β, ν), (29) where T PB(α, β, ν) denotes…”
Learning Deep Sigmoid Belief Networks with Data Augmentation (2015, methodology): “The three parameter beta normal (TPBN) prior (Armagan et al., 2011), a typical global-local shrinkage prior, has demonstrated better (mixing) performance than the aforementioned priors, and thus is employed in this paper.”
Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity (2013, methodology): “This gives the three parameter beta normal mixture priors introduced by Armagan et al. (2011) and is denoted by TPBN(α, β, τ(2)σ(2)).”
A Probabilistic Model for Dirty Multi-task Feature Selection (2015, background): “This distribution is a scale mixture of Gaussians (Armagan et al., 2011) defined as:”