VARIABLE SELECTION IN NONPARAMETRIC ADDITIVE MODELS

By Jian Huang, Joel L. Horowitz, Fengrong Wei. 2010

578 citations recorded by the source corpus.

We consider a nonparametric additive model of a conditional mean function in which the number of variables and additive components may be larger than the sample size but the number of nonzero additive components is “small” relative to the sample size. The statistical problem is to determine which additive components are nonzero. The additive components are approximated by truncated series expansions with B-spline bases. With this approximation, the problem of component selection becomes that of selecting the groups of coefficients in the expansion. We apply the adaptive group Lasso to select nonzero components, using the group Lasso to obtain an initial estimator and reduce the dimension of the problem. We give conditions under which the group Lasso selects a model whose number of components is comparable with the underlying model, and the adaptive group Lasso selects the nonzero components correctly with probability approaching one as the sample size increases and achieves the optimal rate of convergence. The results of Monte Carlo experiments show that the adaptive group Lasso procedure works well with samples of moderate size. A data example is used to illustrate the application of the proposed method.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Fused Lasso Additive Model (2014, methodology): “Flexible additive modeling in high dimensions has been an active area of research in recent years [Avalos et al., 2007, Huang et al., 2010, Li and Liu, 2014, Lin and Zhang, 2006, Sardy and Tseng, 2004, Wood et al., 2014, Zhang et al., 2011].”
Estimation and model selection in generalized additive partial linear models for correlated data with diverging number of covariates (2014, methodology): “After variable screening, we then applied the penalized polynomial splines [13, 32] for high-dimensional additive model selection.”
Oracle Inequalities and Optimal Inference under Group Sparsity (2010, methodology): “Most of them are focused on the Group Lasso for additive models [16, 19, 25, 31] or generalized linear models [24].”
Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain (2012, methodology): “…and Wang, 2011a and 2011b), thresholded Lasso and Post-thresholded Lasso (Belloni and Chernozhukov, 2012), group Lasso and Post-group Lasso (Huang, Horowitz, and Wei, 2010; and Lounici, Pontil, Tsybakov, and van de Geer, 2010), adaptive versions of the above (Huang, Horowitz, and Wei,…”
LASSO Isotone for High-Dimensional Additive Isotonic Regression (2010, methodology): “In minimising predictive error, however, we do still fit some irrelevant covariates as non-zero, a phenomenon previously observed with the LASSO in Leng et al. [2006]. Now, unlike a LARS-like approach, LISO Backfitting will only give us the solution for an individual choice of λ.”
Variable selection for high-dimensional generalized varying-coefficient models (2012, background): “Thus we suggest fixing a relatively large K that is able to approximate the nonparametric functions reasonably well in most situations (for example, in Huang, Horowitz, and Wei (2010) the authors fixed K = 6).”
Trading Interpretability for Accuracy: Oblique Treed Sparse Additive Models (2015, unknown): “LITERATURE REVIEW This section focuses mainly on”
Learning Sparse Additive Models with Interactions in High Dimensions (2016, methodology): “There exist several algorithms for learning these models; we refer to [9, 10, 11, 12, 13] and references therein.”
Group Sparse Additive Models (2012, background): “Finally, it is important to emphasize that our work is substantially diﬀerent from another line of research that also mixes group lasso and additive models (Bach, 2008; Meier et al., 2009; Huang et al., 2010).”
Semiparametric regression models with additive nonparametric components and high dimensional parametric components (2012, background): “Huang et al. (2010) studied variable selection in nonparametric additive models when the number of additive components may diverge with the sample size.”
Program evaluation with high-dimensional data (2013, result): “For a thorough discussion of this assumption, we refer to Imbens and Angrist (1994). Using this assumption, we present the following lemma which follows from results of Abadie (2003) and Hong and Nekipelov (2010) that both build upon the results of Imbens and Angrist (1994).”
Program evaluation and causal inference with high-dimensional data (2013, background): “…Li (2001); Zou (2006); Candès and Tao (2007); van de Geer (2008); Huang et al. (2008); Bickel et al. (2009); Meinshausen and Yu (2009); Bach (2010); Huang et al. (2010); Belloni and Chernozhukov (2011); Kato (2011); Belloni et al. (2012); Belloni and Chernozhukov (2013); Belloni et al. (2013b);…”
GENERALIZED ADDITIVE PARTIAL LINEAR MODELS WITH HIGH-DIMENSIONAL COVARIATES (2013, methodology): “This alternative strategy was also advocated by Huang, Horowitz, and Wei (2010) for removing the requirement of√ n-consistent initial estimator when they studied the adaptive LASSO for variable selection in high-dimensional linear and nonparametric additive models.”
Taking Advantage of Sparsity in Multi-Task Learning (2009, methodology): “Most of them are focused on the Group Lasso for additive models [15, 17, 22, 25] or generalized linear models [21].”
A Selective Review of Group Selection in High-Dimensional Models. (2012, background): “Grouping can also be introduced into a model in the hopes of taking advantage of prior knowledge that is scientifically meaningful.”