GENERALIZED DOUBLE PARETO SHRINKAGE.

By Artin Armagan, David B. Dunson, Jaeyong Lee. Statistica sinica, 2011

398 citations recorded by the source corpus.

We propose a generalized double Pareto prior for Bayesian shrinkage estimation and inferences in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, forming a bridge between the Laplace and Normal-Jeffreys' priors. While it has a spike at zero like the Laplace density, it also has a Student's t-like tail behavior. Bayesian computation is straightforward via a simple Gibbs sampling algorithm. We investigate the properties of the maximum a posteriori estimator, as sparse estimation plays an important role in many problems, reveal connections with some well-established regularization procedures, and show some asymptotic results. The performance of the prior is tested through simulations and an application.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/77a3f0ef7b4b4649d50cebfb45e2fcd438a9480c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Stéphanie van der Pas
2. A. van der Vaart
3. Botond Szab'o
4. Jim E. Griffin
5. David B. Dunson
6. Philip J. Brown
7. Barbara E. Engelhardt
8. Sayan Mukherjee

In Control and Systems Engineering

1. A. van der Vaart
2. Stéphanie van der Pas
3. Botond Szab'o
4. Jim E. Griffin
5. Anirban Bhattacharya
6. Debdeep Pati
7. Malay Ghosh
8. Philip J. Brown

In Statistics and Probability

1. Stéphanie van der Pas
2. Botond Szab'o
3. A. van der Vaart
4. Jim E. Griffin
5. Philip J. Brown
6. Barbara E. Engelhardt
7. Sayan Mukherjee
8. Chong You

Citations in the corpus

• Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “As a comparison, we study the tail behavior of the Dirichlet– Laplace and double Pareto prior.”
• Genetic algorithms: theory, genetic operators, solutions, and applications (2023, neutral): “Thakur [37] presented a new crossover technique called double Pareto crossover (DPX) depending on double Pareto distribution [53].”
• Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity (2013, methodology): “Armagan et al. (2011) introduced the class of “three parameter beta normal” mixture priors while the class of “generalized double Pareto” priors was introduced in Armagan et al. (2012).”
• Dirichlet-Laplace priors for optimal shrinkage (2014, background): “In the regression setting y ∼ Nn(Xβ, In) with the number of covariates pn ≤ n, it is typically assumed that the eigenvalues of (XTX)/n are bounded between two absolute constants (Johnson and Rossell 2012; Armagan et al. 2013).”
• A variational Bayes approach to variable selection (2017, background): “(2010), Griffin and Brown (2011), Armagan et al. (2013) and Neville et al. (2014). Such priors entail at least some shrinkage on the regression coefficients. We believe that some coefficient shrinkage is highly likely to be beneficial in practice. However, we have not pursued such priors here but fo”
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors (2014, background): “Results in (Armagan, Dunson and Lee, 2013) are similar.”
• Bayesian group factor analysis with structured sparsity (2016, methodology): “…have been proposed as a computationally efficient alternative to the two component spike-and-slab prior (West, 1987; Carvalho et al., 2010; Polson and Scott, 2011; Armagan et al., 2013, 2011; Bhattacharya et al., 2014): such priors generally assume normal distributions with a mixed variance term.”
• Polygenic prediction via Bayesian regression and continuous shrinkage priors (2018, background): “…and Doucet, 2008; Griffin and Brown, 2010], the normal-inverse-gaussian prior [Caron and Doucet, 2008], the generalized t (generalized double Pareto) prior [Armagan et al., 2013; Lee et al., 2012], and the normal-exponential-gamma prior [Armagan et al., 2011; Griffin and Brown, 2011], among others.”
• Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies (2014, neutral): “[5] Armagan, A., Dunson, D.B. and Lee, J. (2013).”
• Efficient posterior sampling for high-dimensional imbalanced logistic regression. (2019, background): “[4] Artin Armagan, David B Dunson, and Jaeyong Lee.”
• Hierarchical Shrinkage Priors for Regression Models (2017, background): “, 2012) or double Pareto prior (Armagan et al., 2013) and the exponential power prior (Polson et al.”
• Adaptive posterior contraction rates for the horseshoe (2017, methodology): “The horseshoe posterior has performed well in simulations [7, 6, 23, 22, 3, 1].”
• Uncertainty Quantification for the Horseshoe (with Discussion) (2016, methodology): “The horseshoe prior is popular due to its good performance in simulations and under theoretical study (e.g. Carvalho et al. (2010, 2009); Polson and Scott (2012a, 2010); Bhattacharya et al. (2014); Armagan et al. (2013); van der Pas et al. (2014); Datta and Ghosh (2013)).”
• Generalized Beta Mixtures of Gaussians (2011, background): “This phenomenon has motivated a rich variety of new priors such as the normal-exponential-gamma, the horseshoe and the generalized double Pareto [11, 14, 1, 6, 20, 7, 12, 2].”