Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity

By Tommaso Cai, Zijian Guo. 2015

191 citations recorded by the source corpus.

Confidence sets play a fundamental role in statistical inference. In this paper, we consider confidence intervals for high dimensional linear regression with random design. We first establish the convergence rates of the minimax expected length for confidence intervals in the oracle setting where the sparsity parameter is given. The focus is then on the problem of adaptation to sparsity for the construction of confidence intervals. Ideally, an adaptive confidence interval should have its length automatically adjusted to the sparsity of the unknown regression vector, while maintaining a prespecified coverage probability. It is shown that such a goal is in general not attainable, except when the sparsity parameter is restricted to a small region over which the confidence intervals have the optimal length of the usual parametric rate. It is further demonstrated that the lack of adaptivity is not due to the conservativeness of the minimax framework, but is fundamentally caused by the difficulty of learning the bias accurately.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, methodology): “The functionals we consider are different than those analyzed in Cai and Guo (2017).”
Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, methodology): “1 Approximate Residual Balancing as Debiased Linear Estimation As we have already emphasized, approximate residual balancing is a method that enables us to makes inferences about average treatment effects without needing to estimate treatment propensities as nuisance parameters; rather, we build on ”
A modern maximum-likelihood theory for high-dimensional logistic regression (2018, background): “In the linear model context, [21] established that there is a fundamental limit to the width of confidence intervals that adapt to the unknown sparsity, when the ultra-sparsity condition is relaxed, unless one makes further assumptions regarding the design matrix.”
Optimal inference in a class of regression models (2015, unknown): “See Cai and Guo (2015) for recent work on adaptation to sparsity in high-dimensional regression.”
Semiparametric efficiency bounds for high-dimensional models (2016, methodology): “The statement (1) further requires some mild regularity conditions (see Cai and Guo (2015), Ren et al.”
Local Inference in Additive Models with Decorrelated Local Linear Estimator (2019, background): “In the high-dimensional sparse linear regression, the uncertainty-quantification condition for a single regression coefficient βi has been imposed in [22,39,44] and this condition has shown to be necessary for adaptive inference for a single regression coefficient in [5].”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, methodology): “In high-dimensional literature Gaussian design is a very common assumption (e.g. Javanmard and Montanari (2014b); Cai and Guo (2015)).”
Breaking the curse of dimensionality in regression (2017, background): “Sparsity assumption on the model signals has had a significant role in achieving optimal inference – Cai and Guo (2015); Javanmard and Montanari (2015); Cai and Guo (2016) found minimax results quantifying the direct effect of the size of the sparsity.”
High-dimensional regression adjustments in randomized experiments (2016, background): “Conversely, if we only have k n/ log(p), accurate inference about the coefficients of β is in general impossible without further conditions on the covariance of X [32, 33].”