Penalized regression, standard errors, and Bayesian lassos

By Minjung Kyung, Jeff Gill, Malay Ghosh, George Casella. 2010

644 citations recorded by the source corpus.

Penalized regression methods for simultaneous variable selection and coe-cient estimation, especially those based on the lasso of Tibshirani (1996), have received a great deal of attention in recent years, mostly through frequen- tist models. Properties such as consistency have been studied, and are achieved by difierent lasso variations. Here we look at a fully Bayesian formulation of the problem, which is ∞exible enough to encompass most versions of the lasso that have been previously considered. The advantages of the hierarchical Bayesian for- mulations are many. In addition to the usual ease-of-interpretation of hierarchical models, the Bayesian formulation produces valid standard errors (which can be problematic for the frequentist lasso), and is based on a geometrically ergodic Markov chain. We compare the performance of the Bayesian lassos to their fre- quentist counterparts using simulations, data sets that previous lasso papers have used, and a di-cult modeling problem for predicting the collapse of governments around the world. In terms of prediction mean squared error, the Bayesian lasso performance is similar to and, in some cases, better than, the frequentist lasso.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/717ce0992d640e385ae389f353e82c198b96958b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

The Bayesian Lasso (2008, methodology): “This example from Park and Casella (2008) fits a Bayesian LASSO model to the diabetes data from Efron et al.”
Machine Learning for Survival Analysis (2019, methodology): “Penalized Regression: Penalized regression methods (Kyung et al. 2010) are best known for their useful property of simultaneous variable selection and coefficient estimation.”
A guide to Bayesian model selection for ecologists (2015, methodology): “Unlike in a Metropolis-Hastings MCMC algorithm, the resulting Gibbs sampler requires no tuning of any parameters (Kyung et al. 2010).”
Hierarchical Shrinkage Priors for Regression Models (2017, methodology): “More generally, a Bayesian version of the group Lasso (Yuan and Lin, 2006) was developed by Kyung et al. (2010) and Raman et al. (2009).”
Bayesian group factor analysis with structured sparsity (2016, positive): “However, there has been little work in using Bayesian structured sparsity, with some exceptions (Kyung et al., 2010; Engelhardt and Adams, 2014)).”
Geometric ergodicity of the Bayesian lasso (2013, background): “[12] contains, among other results, a proof of geometric ergodicity of the Bayesian lasso Gibbs Markov chain.”
Hierarchical shrinkage priors for dynamic regressions with many predictors (2011, positive): “Thus, this estimator penalizes the L 1 -norm of both the coefﬁcients and their differences The representation of the Bayesian prior for (cid:12) in the penalized regression using the Fused LASSO is (cid:0) (cid:1) Kyung et al. (2010) show that the hierarchical representation of this prior is”
Review and evaluation of penalised regression methods for risk prediction in low‐dimensional data with few events (2015, background): “[16], who did not present any coverage results in their simulation studies, and more recently by Efron [17].”
Machine Learning for Survival Analysis: A Survey (2017, methodology): “Penalized Regression: Penalized regression methods [Kyung et al. 2010] are well-known for their nice properties of simultaneous variable selection and coefﬁcient estimation.”