Confidence sets in sparse regression

By Richard Nickl, Sara A. van de Geer. 2012

113 citations recorded by the source corpus.

The problem of constructing confidence sets in the high-dimensional linear model with $n$ response variables and $p$ parameters, possibly $p\ge n$, is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed $n^{-1/4}$, otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that adapt to a fixed sparsity level of the parameter vector are given in terms of minimal $\ell^2$-separation conditions on the parameter space. The design conditions cover common coherence assumptions used in models for sparsity, including (possibly correlated) sub-Gaussian designs.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/fed8c23c633ce48249d5cbd8f5f22da179357aa9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Bootstrapping and sample splitting for high-dimensional, assumption-lean inference (2016, background): “, [14], [76], [36], [15], [26], [77], [72], [55].”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, background): “It follows from general results of [23, 33, 27] that honest uncertainty quantification is irreconcilable with adaptation to sparsity.”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, background): “Consider the case of Σ X = cI p for some c > 0 ; a prototypical model in compressive sensing corresponds to c = 1 (Nickl et al., 2013).”
Hypothesis Testing For Densities and High-Dimensional Multinomials: Sharp Local Minimax Rates (2017, background): “Finally, it would also be interesting to further explore the extent to which the local-minimax perspective can lead to a better understanding of composite-null inference problems [1, 2, 5, 8, 10, 23, 30].”
On the uniform convergence of empirical norms and inner products, with application to causal inference (2013, methodology): “Following an idea of [16] (see also Lemma 1 in [22]) one can show that the supremum over all ‖fβ‖ ≤ 1 can in fact be replaced by a maximum over a finite class:”
Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity (2015, methodology): “Such has been considered in [19], where the impossibility of adaptive confidence balls for sparse linear regression was established.”
Asymptotically Honest Confidence Regions for High Dimensional Parameters by the Desparsified Conservative Lasso (2014, background): “Nickl and van de Geer (2013) consider honest adaptive inference when p > n. This can be obtained as long as the rate of sparse estimation does not exceed n−1/4. Hoffmann and Nickl (2011) consider the existence of honest adaptive confidence bands for an unknown density function. They show that this i”
High-dimensional econometrics and regularized GMM (2018, background): “[97] also consider the construction of confidence sets for the entire parameter vector in a sparse high-dimensional regression using sample splitting ideas.”
Robust machine learning by median-of-means: Theory and practice (2017, background): “Even if recent advances [98, 91, 74] have shown some limitations of LASSO, it remains the benchmark estimator in high-dimensional statistics because a high dimensional parameter space does not significantly affect its performance as long as t∗ is sparse.”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “At a theoretical level, the paper [35] derives confidence intervals in 2 for the case of two possible sparsity levels.”
Confidence intervals for high-dimensional inverse covariance estimation (2014, background): “We note that existing work on statistical inference in high dimensional settings has mostly focused on inference for parameters in linear models and generalized linear models [15, 17, 8, 6, 3, 2, 10].”
Convergence rates of least squares regression estimators with heavy-tailed errors (2017, background): “Previous work guaranteeing rate-optimality of the Lasso estimator typically assumes that both X and ξ are sub-Gaussian; see [14, 51, 62].”