Tight conditions for consistency of variable selection in the context of high dimensionality

By Laëtitia Comminges, Arnak S. Dalalyan. 2011

70 citations recorded by the source corpus.

We address the issue of variable selection in the regression model with very high ambient dimension, that is, when the number of variables is very large. The main focus is on the situation where the number of relevant variables, called intrinsic dimension, is much smaller than the ambient dimension d. Without assuming any parametric form of the underlying regression function, we get tight conditions making it possible to consistently estimate the set of relevant variables. These conditions relate the intrinsic dimension to the ambient dimension and to the sample size. The procedure that is provably consistent under these tight conditions is based on comparing quadratic functionals of the empirical Fourier coefficients with appropriately chosen threshold values. The asymptotic analysis reveals the presence of two quite different re gimes. The first regime is when the intrinsic dimension is fixed. In this case the situation in nonparametric regression is the same as in linear regression, that is, consistent variable selection is possible if and only if log d is small compared to the sample size n. The picture is different in the second regime, that is, when the number of relevant variables denoted by s tends to infinity as $n\to\infty$. Then we prove that consistent variable selection in nonparametric set-up is possible only if s+loglog d is small compared to log n. We apply these results to derive minimax separation rates for the problem of variable

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d356a70c2dfcd5201a1e4d136387c1df4ea75139

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Faithful Variable Screening for High-Dimensional Convex Regression (2014, methodology): “Comminges and Dalalyan (2012) have shown that under tradtional smoothness constraints, even with a product distribution, variable selection is achievable only if n > O(es0).”
Support and Invertibility in Domain-Invariant Representations (2019, background): “Theorem 4 (Variable selection in non-parametric regression (Comminges et al. , 2012)) .”
Minimax Testing of a Composite null Hypothesis Defined via a Quadratic Functional in the Model of regression (2012, unknown): “One example, closely related to the problem of variable selection (Comminges and Dalalyan, 2012), is testing the relevance of a particular covariate in high-dimensional regression.”
Learning Sparse Additive Models with Interactions in High Dimensions (2016, unknown): “This model was also studied in [34, 35] in the regression setting – they proposed an estimator that recovers S w.”
Nonparametric sparsity and regularization (2012, background): “More recently, an estimator based on the comparison of some well chosen empirical Fourier coefficients to a prescribed significance level is described and studied in [20] where a careful statistical analysis is proposed considering different regimes for n, d and d∗, where d∗ is the cardinality of Rρ”