Inference with normal-gamma prior distributions in regression problems

By Jim E. Griffin, Philip J. Brown. 2010

529 citations recorded by the source corpus.

This paper considers the efiects of placing an absolutely continuous prior distribution on the regression coe-cients of a linear model. We show that the posterior expectation is a matrix-shrunken version of the least squares estimate where the shrinkage matrix depends on the derivatives of the prior predictive den- sity of the least squares estimate. The special case of the normal-gamma prior, which generalizes the Bayesian Lasso (Park and Casella 2008), is studied in depth. We discuss the prior interpretation and the posterior efiects of hyperparameter choice and suggest a data-dependent default prior. Simulations and a chemomet- ric example are used to compare the performance of the normal-gamma and the Bayesian Lasso in terms of out-of-sample predictive performance.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b71c3c5b9f7ee4d76b8558c2baedf3843d1d93c1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Geometric ergodicity of the Bayesian lasso (2013, methodology): “have been introduced and studied in the literature (see [4, 5, 6, 10, 15] and the references therein), and sampling from the resulting posterior distribution is often achieved by using MCMC.”
Moving beyond noninformative priors: why and how to choose weakly informative priors in Bayesian analyses (2019, background): “This prior also requires careful consideration of the scale of both the response and predictors, and is a generalized version of frequentist LASSO regression (Park and Casella 2008, Griffin and Brown 2010).”
Comparing distributions by using dependent normalized random‐measure mixtures (2013, methodology): “In our applications, we shall mostly focus on an alternative approach and define a prior for Mh which encourages substantial shrinkage towards zero (this is similar to the shrinkage prior approach to regression described by Scott and Polson (2011) and Griffin and Brown (2010)).”
Scalable Variational Inference for Bayesian Variable Selection in Regression, and Its Accuracy in Genetic Association Studies (2012, background): “…of the prior, so it may be possible to extend it to non-sparse priors that induce shrinkage in the regression coefficients, including normal-gamma priors (Griffin and Brown 2010) and the Bayesian Lasso (Park and Casella 2008), but investigating this question lies outside the scope of this paper.”
Sparse Bayesian time-varying covariance estimation in many dimensions (2016, methodology): “For more elaborate discussions on Bayesian shrinkage in general and the effect of ai specifically, see Griffin and Brown (2010) and Polson and Scott (2011). Note that the Bayesian Lasso prior (Park and Casella 2008) arises as a special case when ai = 1. One can see prior (4) as row-wise shrinkage wi”
BAYESIAN LINEAR REGRESSION WITH SPARSE PRIORS (2014, unknown): “See for instance [15, 8, 16].”
Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction (2012, background): “Normal/gamma and normal/inverse-Gaussian, respectively characterized by gamma and inverse-Gaussian mixing densities [Caron and Doucet, 2008, Griffin and Brown, 2010].”
Bayesian quantile regression for ordinal longitudinal data (2016, background): “However, this prior performs poorly if there are big differences in the size of fixed effects [15].”
Model Averaging and Its Use in Economics (2017, background): “Examples are the normalgamma prior of Griffin and Brown (2010) and the horseshoe prior of Carvalho et al. (2010).”
Adaptive posterior contraction rates for the horseshoe (2017, background): “Those most directly related to this work are [30, 20, 9, 8, 18, 16, 19, 14, 5, 3, 2, 25].”
Polygenic prediction via Bayesian regression and continuous shrinkage priors (2018, background): “the class of continuous shrinkage priors, which includes the normal-gamma prior [Caron and Doucet, 2008; Griffin and Brown, 2010], the normal-inverse-gaussian prior [Caron and Doucet, 2008], the generalized t (generalized double Pareto) prior [Armagan et al.”
BAYESIAN HYPER‐LASSOS WITH NON‐CONVEX PENALIZATION (2011, methodology): “This has been used in a fully Bayesian analysis for regression problems by Griffin & Brown (2010).”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, background): “…structures Tibshirani (1996); Johnstone and Silverman (2004); Castillo and Van der Vaart (2012); Castillo et al. (2015); Jiang and Zhang (2009); Griffin and Brown (2010); Johnson and Rossell (2010); Ghosh and Chakrabarti (2015); Caron and Doucet (2008); Bhattacharya et al. (2014); Bhadra et…”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, methodology): “Default choices of the hyper-parameters ν and μ follow from the recommendations of Griffin and Brown (2010) and are discussed in the implementation in Section 6.”