Structured variable selection and estimation

By Ming Yuan, V. Joseph, Hui Zou. 2009

121 citations recorded by the source corpus.

In linear regression problems with related predictors, it is desirable to do variable selection and estimation by maintaining the hierarchical or structural relationships among predictors. In this paper we propose non-negative garrote methods that can naturally incorporate such relationships defined through effect heredity principles or marginality principles. We show that the methods are very easy to compute and enjoy nice theoretical properties. We also show that the methods can be easily extended to deal with more general regression problems such as generalized linear models. Simulations and real examples are used to illustrate the merits of the proposed methods.

Subfields: Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/408eb55adaee1aee680a23210e7cfe5510e87677

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

A LASSO FOR HIERARCHICAL INTERACTIONS (2012, methodology): “…through a prior [Chipman (1996)]; and, most related to this paper’s proposal, regularized regression methods, which are deﬁned by an optimization problem [Yuan, Joseph and Zou (2009), Zhao, Rocha and Yu (2009), Choi, Li and Zhu (2010), Jenatton et al. (2010), Radchenko and James (2010)].”
SLEP: Sparse Learning with Efficient Projections (2011, methodology): “The ordered tree structure is a special case of the non-negative garrote discussed in [79] when the hierarchical relationship is depicted by a tree.”
Hierarchical Shrinkage Priors for Regression Models (2017, methodology): “…implementations of spike-and-slab priors as in (Chipman, 1996) and some citations above; (ii) use of Bayesian tools unlike Bien et al. (2013) or Yuan et al. (2009); (iii) use of flexible and possibly heavy tailed priors, unlike Chipman et al. (1997) or Yi et al. (2007); (iv) providing a simple…”
Identification of Partially Linear Structure in Additive Models with an Application to Gene Expression Prediction from Sequences (2012, background): “…the introduction of lasso in Tibshirani (1996), the penalization approach for variable selection has become increasingly popular (Zou and Hastie, 2005; Zhao and Yu, 2006; Zhang and Huang, 2008; Bickel, Ritov, and Tsybakov, 2009; Wu and Liu, 2009; Yuan, Joseph, and Zou, 2009; Peng et al., 2010).”