The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

By Stéphanie van der Pas, B. Kleijn, A. van der Vaart. 2014

191 citations recorded by the source corpus.

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax $\ell_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/16371329845aeddd3e5f5c8b45cd8d71629ff30c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Uncertainty Quantification for the Horseshoe (with Discussion) (2016, methodology): “We study the relative performances of the empirical Bayes and hierarchical Bayes approaches further through simulation studies, extending the simulation study in [49].”
Comparison of Bayesian predictive methods for model selection (2015, background): “For instance the Horseshoe prior (Carvalho et al., 2009, 2010) has been shown to have desirable properties empirically and theoretically assuming a properly chosen shrinkage factor (Datta and Ghosh, 2013; van der Pas et al., 2014).”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “In the normal means model, van der Pas, Kleijn, and van der Vaart (2014) and van der Pas, Szabó, and van der Vaart (2017a) investigated the Horseshoe posterior contraction rate, Bhattacharya et al. (2015) showed the optimal posterior concentration results for Dirichlet–Laplace prior, Bhadra et al.…”
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, background): “Of the 400 predictors, only the ﬁrst three have nonzero coeﬃcients β 1 , 2 , 3 = (3 , 2 , 1); this is the example discussed in Van Der Pas et al. (2014) and Piironen and Vehtari (2017b).”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “Maximum marginal likelihood estimator In this Section we define the MMLE and compare it to a naive empirical Bayes estimator previously suggested in [33].”
The Spike-and-Slab LASSO (2018, unknown): “For more background on rates of posterior concentration in regression settings, we refer the reader to Castillo, Schmidt-Hieber, and van der Vaart (2015); Martin and Walker (2014); van der Pas, Kleijn, and van der Vaart (2014); Bhattacharya et al. (2015); and Rockova (2015).”
Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity (2013, methodology): “For one part, as noted by van der Pas et al. (2014), we use the fact that, for each fixed x ∈ R, E(1−κi|x, τ) is non-decreasing in τ , and then we use the results of Theorem 6 and Theorem 7, while for the other part, we exploit the structure of τ̂ , together with the independence among the Xi’s.”
The Graphical Horseshoe Estimator for Inverse Covariance Matrices (2017, background): “van der Pas et al. (2017) studied the coverage properties of marginal credible intervals under the horseshoe prior, for a sparse normal means problem.”