Dirichlet-Laplace priors for optimal shrinkage

By Anirban Bhattacharya, Debdeep Pati, Natesh S. Pillai, David B. Dunson. Journal of the American Statistical Association, 2014

479 citations recorded by the source corpus.

Penalized regression methods, such as L1 regularization, are routinely used in high-dimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through two-component mixture priors having a probability mass at zero, but such priors encounter daunting computational problems in high dimensions. This has motivated continuous shrinkage priors, which can be expressed as global-local scale mixtures of Gaussians, facilitating computation. In contrast to the frequentist literature, little is known about the properties of such priors and the convergence and concentration of the corresponding posterior distribution. In this article, we propose a new class of Dirichlet–Laplace priors, which possess optimal posterior concentration and lead to efficient posterior computation. Finite sample performance of Dirichlet–Laplace priors relative to alternatives is assessed in simulated and real data examples.

Subfields: Civil and Structural Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/91da74f48a1525beddbedd6177f76ebf8ef12c39

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Civil and Structural Engineering

In Statistics and Probability

Citations in the corpus

The Spike-and-Slab LASSO (2018, unknown): “For more background on rates of posterior concentration in regression settings, we refer the reader to Castillo, Schmidt-Hieber, and van der Vaart (2015); Martin and Walker (2014); van der Pas, Kleijn, and van der Vaart (2014); Bhattacharya et al. (2015); and Rockova (2015).”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, methodology): “Despite the ubiquity of problems with sparse structures, and the large amount of research effort into finding consistent and minimax optimal estimators for the underlying sparse structures [44, 22, 13, 12, 20, 18, 21, 15, 8, 5, 4, 34], the number of options for uncertainty quantification in the spar”
Bayesian group factor analysis with structured sparsity (2016, methodology): “Recently, scale mixtures of normal priors have been proposed as a computationally efficient alternative to the two component spike-and-slab prior (West, 1987; Carvalho et al., 2010; Polson and Scott, 2011; Armagan et al., 2013, 2011; Bhattacharya et al., 2014).”
Sparsity information and regularization in the horseshoe and other shrinkage priors (2017, background): “As a final remark, although we focus on the horseshoe in our discussion, we want to emphasize that both of these ideas could also be applied to other shrinkage priors, and several promising alternatives to the horseshoe have been proposed during the recent years (Bhattacharya et al., 2015; Zhang, Re”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “ude normal-gamma mixtures (Grin et al., 2010), Horseshoe (Carvalho et al., 2009, 2010), generalized Beta (Armagan et al., 2011), generalized double Pareto (Armagan et al., 2013a), Dirichlet-Laplace (Bhattacharya et al., 2015), and Horseshoe+ (Bhadra et al., 2016). Global-local priors have substantia”
Adaptive Shrinkage in Bayesian Vector Autoregressive Models (2019, background): “Moreover, the SSVS prior, although possessing convenient theoretical properties (Bhattacharya et al. 2015), imposes a severe computational burden if the model space to explore is large.”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “The horseshoe posterior has performed well in simulations [7, 6, 23, 22, 3, 1].”
The Graphical Horseshoe Estimator for Inverse Covariance Matrices (2017, background): “, 2017) or the Dirichlet–Laplace (Bhattacharya et al., 2015) should also be explored.”