Local shrinkage rules, Lévy processes and regularized regression

By Nicholas G. Polson, James G. Scott. 2010

142 citations recorded by the source corpus.

Summary.  We use Lévy processes to generate joint prior distributions, and therefore penalty functions, for a location parameter as p grows large. This generalizes the class of local–global shrinkage rules based on scale mixtures of normals, illuminates new connections between disparate methods and leads to new results for computing posterior means and modes under a wide class of priors. We extend this framework to large‐scale regularized regression problems where p>n, and we provide comparisons with other methodologies.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8c23fa384dfaf53346811bf310105db6575f571c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lawrence Carin
2. Zhe Gan
3. Jim E. Griffin
4. Chunyuan Li
5. Ricardo Henao
6. Philip J. Brown
7. Brian J. Reich
8. B. Naughton

In Statistics and Probability

1. Jim E. Griffin
2. Philip J. Brown
3. Nicholas G. Polson
4. Bruce A. Craig
5. Anindya Bhadra
6. Brian J. Reich
7. B. Naughton
8. Y. Zhang

Citations in the corpus

• Proximal Algorithms in Statistics and Machine Learning (2015, methodology): “…processing [Geman and Reynolds, 1992, Geman and Yang, 1995, Rudin et al., 1992], nonlinear curve fitting [Tibshirani, 2014], Bayesian MAP inference [Polson and Scott, 2012], multiple hypothesis testing [Tansey et al., 2014] and shrinkage/sparsity-inducing prior regularisation problems [Green et…”
• The Graphical Horseshoe Estimator for Inverse Covariance Matrices (2017, methodology): “Although the SCAD estimate cannot be expressed as a maximum a posteriori estimate, the prior density corresponding to the SCAD penalty would be bounded by Theorem 1 in Polson and Scott (2012).”
• Learning Deep Sigmoid Belief Networks with Data Augmentation (2015, background): “Some canonical examples are the spike-and-slab prior, the Student's-t prior, the double exponential prior and the horseshoe prior (see Polson and Scott (2012) for a discussion of these priors).”
• Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, methodology): “According to Raskutti et al. (2011), the minimax L2 rate is O( √ qn log(pn/qn)/n).”
• Hierarchical Shrinkage Priors for Regression Models (2017, methodology): “Results for linear regression models which express the posterior expectation and variance in terms of the least squares estimate of β and the variance of its sampling distribution (for n > p) have been derived by several authors including Griffin and Brown (2010) and Polson and Scott (2012).”
• Journal of the American Statistical Association Bayesian Inference for Logistic Models Using Pólya–gamma Latent Variables Bayesian Inference for Logistic Models Using P ´ Olya–gamma Latent Variables (unknown): “This follows the Lévy-penalty framework of Polson and Scott (2012) and relates to work by Ciesielski and Taylor (1962) on the sojourn times of Brownian motion.”