Handling Sparsity via the Horseshoe

By Carlos M. Carvalho, Nicholas G. Polson, James G. Scott. 2009

625 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Survey on Multi-Task Learning (2017, background): “When C becomes an identity matrix which means that features are independent, this prior degenerates to the horseshoe prior [36] which can induce sparse estimations.”
Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “…in machine learning (MacKay, 1992; Hinton & van Camp, 1993; Buntine & Weigend, 1991; Neal, 1995; De Freitas, 2003) and Mnih & Gregor (2014), where a neural network is used in a variational approximation to the posterior distribution over the latent variables of a directed generative neural network.”
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors (2020, background): “scale induces a horseshoe distribution (Carvalho et al., 2009).”
Bayesian group factor analysis with structured sparsity (2016, neutral): “…regularization, across all coefficients, and local regularization, specific to each coefficient (Polson and Scott, 2011))—has behavior that is similar to the spike-and-slab prior in effectively modeling signal and noise separately, but with computational tractability (Carvalho et al., 2009).”
Smart Regularization of Deep Architectures (2015, methodology): “For example we could employ a Horsehoe distribution [48], an Automatic Relevance Determination [49, 50] framework, or a Bayesian Lasso [51] framework.”
Dirichlet-Laplace priors for optimal shrinkage (2014, background): “However, if one instead uses a half Cauchy prior ψ j ∼ Ca+(0, 1), then the resulting horseshoe kernel [6, 7] is unbounded with a singularity at zero.”
Bayesian Optimization and Semiparametric Models with Applications to Assistive Technology (2014, background): “As we expect the observation noise generally to be close to or exactly zero, ν is given a horseshoe prior [Carvalho et al., 2009].”
Encoding Tree Sparsity in Multi-Task Learning: A Probabilistic Framework (2014, methodology): “The Cauchy prior is widely used for feature learning (Carvalho, Polson, and Scott 2009; Daniel, Jose Miguel, and Pierre 2013).”
BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery (2021, methodology): “The horseshoe prior has a sharp peak at zero and relatively flat tails which tend to induce sparsity in the posterior while not significantly penalizing larger coefficients [7].”
Geometric ergodicity of the Bayesian lasso (2013, background): “In recent years, many other continuous shrinkage priors have been introduced and studied in the literature (see [4, 5, 6, 10, 15] and the references therein), and sampling from the resulting posterior distribution is often achieved by using MCMC.”
Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors (2018, background): “…Louizos et al. (2017) introduce a prior such that each unit’s weight vector w kl is conditionally independent and follow a group Horseshoe prior (Carvalho et al., 2009), (1) Here, I is an identity matrix, a ∼ C + (0 , b ) is the Half-Cauchy distribution with density p ( a b ) = 2 /πb (1 + ( a…”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, methodology): “This paper studies the Bayesian approach based on the horseshoe prior [10, 9, 37, 30, 31].”
Model Selection in Bayesian Neural Networks via Horseshoe Priors (2017, methodology): “The distribution over weights in Equation 2 is called horseshoe prior [5].”
GENERALIZED DOUBLE PARETO SHRINKAGE. (2011, methodology): “The horseshoe prior of Carvalho, Polson, and Scott (2009, 2010) is induced through a carefully-specified mixture of normals, leading to such desirable properties as an infinite spike at zero and very heavy tails.”
Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity (2013, methodology): “Note that the above estimator of τ is truncated below by 1m and hence it is not susceptible to collapsing to zero, which is a major concern for the use of such empirical Bayes approaches as mentioned in Carvalho et al. (2009), Scott and Berger (2010), Bogdan et al. (2008) and Datta and Ghosh (2013).”
Generalized Beta Mixtures of Gaussians (2011, methodology): “This phenomenon has motivated a rich variety of new priors such as the normal-exponential-gamma, the horseshoe and the generalized double Pareto [11, 14, 1, 6, 20, 7, 12, 2].”