BAYESIAN LINEAR REGRESSION WITH SPARSE PRIORS

By I. Castillo, Johannes Schmidt-Hieber, A. van der Vaart. 2014

414 citations recorded by the source corpus.

We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6ba008f337dbb426882fe4a86acba9aaf1dfacb9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates (2020, background): “Quoted Descloux and Sardy (2018): although diﬀering in their purposes and performance, the general idea underlying these procedures remains the same, namely to avoid overﬁtting by ﬁnding a trade-oﬀ between the ﬁt y − Xβ and some measure of the model complexity.”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “Those most directly related to this work are [30, 20, 9, 8, 18, 16, 19, 14, 5, 3, 2, 25].”
Model Averaging and Its Use in Economics (2017, background): “In particular, Castillo et al. (2015) prove that BMA in linear regression leads to an optimal rate of contraction of the posterior on the regression coefficients to a sparse “true” data-generating model (a model where many of the coefficients are zero), provided the prior sufficiently penalizes…”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, methodology): “…optimal estimators for the underlying sparse structures Tibshirani (1996); Johnstone and Silverman (2004); Castillo and Van der Vaart (2012); Castillo et al. (2015); Jiang and Zhang (2009); Griffin and Brown (2010); Johnson and Rossell (2010); Ghosh and Chakrabarti (2015); Caron and Doucet…”
Adaptive Bernstein–von Mises theorems in Gaussian white noise (2014, neutral): “: 10 3.2 Slab and spike prior in L1 Consider the slab and spike prior, whose frequentist contraction rate has been analyzed in Castillo and van der Vaart [11], Homann et al. [19] and Castillo et al. [10]. The assumptions in [19] ensure that prior draws are very sparse and only very few coecients are”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “Note that these rates are the same as the rates achieved by spike-and-slab approaches as in Castillo et al. (2015), Song & Liang (2017) and Ročková & George (2018).”
The Spike-and-Slab LASSO (2018, methodology): “With λ 0 → ∞ , the SSL prior approaches the point-mass mixture prior, which was shown to yield optimal posterior concentration (Castillo, Schmidt-Hieber, and van der Vaart 2015).”
On the Frequentist Properties of Bayesian Nonparametric Methods (2016, background): “(Castillo et al., 2015) have extended the results of (Castillo and van der Vaart, 2012) to the case of high dimensional linear regression, with a prior distribution on the sparsity inducing very sparse models.”
A variational Bayes approach to variable selection (2017, background): “For example, [13, 44, 3], and [14] showed that the convergence to the true model is power of p or at an exponential rate in log p, where the number of variables p is allowed to grow exponentially in n.”