Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction

By Nicholas G. Polson, James G. Scott, Bertrand S. Clarke, C. Severinski. 2012

541 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/4e56ab1afd8a515a0a0b351fbf1b1d08624d0cc2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Comparing distributions by using dependent normalized random‐measure mixtures (2013, methodology): “In our applications, we shall mostly focus on an alternative approach and define a prior for Mh which encourages substantial shrinkage towards zero (this is similar to the shrinkage prior approach to regression described by Scott and Polson (2011) and Griffin and Brown (2010)).”
Sparse Bayesian time-varying covariance estimation in many dimensions (2016, background): “nkage towards zero, while choosing a i large imposes little shrinkage. For more elaborate discussions on Bayesian shrinkage in general and the eect of a i specically, see Grin and Brown (2010) and Polson and Scott (2011). Note that the Bayesian Lasso prior (Park and Casella2008) arises as a special ”
Adaptive posterior contraction rates for the horseshoe (2017, methodology): “The horseshoe posterior has performed well in simulations [7, 6, 23, 22, 3, 1].”
Sparsity information and regularization in the horseshoe and other shrinkage priors (2017, methodology): “Given the similarity of the shrinkage profiles between the horseshoe and spike-and-slab, it is not surprising that the two priors have shown comparable performance in a variety of experiments (Carvalho, Polson and Scott, 2009, 2010; Polson and Scott, 2011).”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, background): “, 2009), an example of a global-local shrinkage prior (Polson and Scott, 2010), that in addition to alleviating the inflated predictive uncertainty issue allows us to recover smaller networks.”
On the half-cauchy prior for a global scale parameter (2011, background): “There is a large literature on the choice of p(u(2)i ), with Polson and Scott (2011a) providing a recent review.”
Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, background): “och (1993). These issues with discrete mixture priors motivate continuous shrinkage priors. The shrinkage priors are essentially written as global-local scale mixture Gaussian family as summarized in Polson and Scott (2010), i.e., 2 j j˚ j;!˘N(0;!˚ j); ˚ j ˘ˇ(˚ j); (!;˙2) ˘ˇ(!;˙); where ! represents”
Polygenic prediction via Bayesian regression and continuous shrinkage priors (2018, methodology): “First, in contrast to fixing the global shrinkage parameter φ in the model based on prior beliefs about the sparsity of the genetic architecture, or searching a small number of grid values, the global parameter could be learnt from data using empirical Bayes or a full Bayesian approach by placing, f”
GENERALIZED DOUBLE PARETO SHRINKAGE. (2011, background): “As noted in Polson and Scott (2010), if π(τ) has exponential or lighter tails, observations are shrunk towards zero by some non-diminishing amount, regardless of size.”
Hierarchical Shrinkage Priors for Regression Models (2017, background): ““Two group” priors, as classified by Polson and Scott (2011), assume that G is a discrete mixing distribution with two possible values.”
Adaptive Shrinkage in Bayesian Vector Autoregressive Models (2019, background): “Second, a Gamma prior on the coefficients induces a fat-tailed marginal prior ensuring that non-zero signals are not too strongly put to zero (Polson and Scott, 2010).”
Rao–Blackwellization for Bayesian Variable Selection and Model Averaging in Linear and Binary Regression: A Novel Data Augmentation Approach (2011, background): “Scale mixtures of normal distributions have also seen a resurgence of interest for shrinkage estimators (priors with no explicit probability at zero) (Carvalho, Polson, and Scott 2010; Polson and Scott 2010) where MAP estimates are easy to obtain.”
Dirichlet-Laplace priors for optimal shrinkage (2014, background): “Polson and Scott (2010) noted that essentially all such shrinkage priors can be represented as global-local (GL) mixtures of Gaussians, θj ∼ N(0, ψj τ ), ψj ∼ f, τ ∼ g, (5) where τ controls global shrinkage toward the origin while the local scales {ψj } allow deviations in the degree of shrinkage.”