Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences

By I. Castillo, A. van der Vaart. 2012

269 citations recorded by the source corpus.

We consider full Bayesian inference in the multivariate normal mean model in the situation that the mean vector is sparse. The prior distribution on the vector of means is constructed hierarchically by first choosing a collection of nonzero means and next a prior on the nonzero values.We consider the posterior distribution in the frequentist set-up that the observations are generated according to a fixed mean vector, and are interested in the posterior distribution of the number of nonzero components and the contraction of the posterior distribution to the true mean vector. We find various combinations of priors on the number of nonzero coefficients and on these coefficients that give desirable performance. We also find priors that give suboptimal convergence, for instance, Gaussian priors on the nonzero coefficients.We illustrate the results by simulations. © 2012 Institute of Mathematical Statistics.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/35407d9652fd78e4f783c2559f30bd7dd66051cd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors (2014, methodology): “Chib and Greenberg (1998) suggest using a multivariate Gaussian prior for R restricted to the subset where R is positive definite.”
The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors (2014, methodology): “The failure of the Lasso to concentrate around the true mean vector at the minimax rate does indicate that heavy tails in itself may not be suﬃcient, and adding mass at zero solves this problem (Castillo, Schmidt-Hieber and Van der Vaart, 2014; Castillo and Van der Vaart, 2012).”
Thomas Bayes’ walk on manifolds (2012, methodology): “Proof We use a general approach to prove lower bounds for posterior measures introduced in [6] (see [6], [7] for examples).”
Adaptive Bernstein–von Mises theorems in Gaussian white noise (2014, methodology): “The author would like to thank Richard Nickl, Aad van der Vaart, Johannes Schmidt-Hieber, the Associate Editor and two referees for their valuable comments.”
A variational Bayes approach to variable selection (2017, background): “For example, [13, 44, 3], and [14] showed that the convergence to the true model is power of p or at an exponential rate in log p, where the number of variables p is allowed to grow exponentially in n.”
BAYESIAN LINEAR REGRESSION WITH SPARSE PRIORS (2014, methodology): “The paper [16] contains a theoretical analysis similar to the present paper, but restricted to the special case that the regression matrix X is the identity and p = n; see Example 1.”
Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector (2013, methodology): “These include the lasso (Tibshirani 1996), the Bayesian lasso (Park and Casella 2008), the horseshoe prior estimator (Carvalho et al. 2010), the empirical Bayes estimators of Jiang and Zhang (2009), Brown and Greenshtein (2009), and, most recently, Koenker and Mizera (2014).”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “Those most directly related to this work are [30, 20, 9, 8, 18, 16, 19, 14, 5, 3, 2, 25].”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, background): “Despite the ubiquity of problems with sparse structures, and the large amount of research effort into finding consistent and minimax optimal estimators for the underlying sparse structures [44, 22, 13, 12, 20, 18, 21, 15, 8, 5, 4, 34], the number of options for uncertainty quantification in the spar”