Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach

By Victor Chernozhukov, Christian Hansen, Martin Spindler. 2015

190 citations recorded by the source corpus.

We present an expository, general analysis of valid post-selection or post-regularization inference about a low-dimensional target parameter in the presence of a very high-dimensional nuisance parameter that is estimated using selection or regularization methods. Our analysis provides a set of high-level conditions under which inference for the low-dimensional parameter based on testing or point estimation methods will be regular despite selection or regularization biases occurring in the estimation of the high-dimensional nuisance parameter. A key element is the use of so-called immunized or orthogonal estimating equations that are locally insensitive to small mistakes in the estimation of the high-dimensional nuisance parameter. As an illustration, we analyze affine-quadratic models and specialize these results to a linear instrumental variables model with many regressors and many instruments. We conclude with a review of other developments in post-selection inference and note that many can be viewed as...

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, positive): “…[Zhang and Zhang (2014); Belloni et al. (2011, 2014, 2015); Javanmard and Montanari (2014a,b, 2018); Van de Geer et al. (2014); Ning and Liu (2017); Chernozhukov et al. (2015); Neykov et al. (2018); Ren et al. (2015); Jankova and Van De Geer (2015, 2016, 2018); Bradic and Kolar (2017); Zhu and…”
Double/Debiased/Neyman Machine Learning of Treatment Effects (2017, methodology): “Using moment conditions that satisfy (1.4) is also crucial to developing valid inference procedures for θ0 after using ML methods to produce estimators η̂0 as discussed, e.g., in Chernozhukov, Hansen, and Spindler (2015).”
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, methodology): “In general, the challenge in using adaptive methods as the basis for valid statistical inference is that selection bias can be difficult to quantify; see Berk et al. (2013), Chernozhukov, Hansen, and Spindler (2015), Taylor and Tibshirani (2015), and references therein for recent advances.”
The Impact of Machine Learning on Economics (2018, methodology): “An early example of the application of ML methods to causal inference (see Belloni et al. (2014) and Chernozhukov et al. (2015) for reviews) concerns the question of estimating the average eﬀect of a treatment in an environment with unconfoundedness, where the treatment assignment is as good as…”
Double/debiased machine learning for treatment and structural parameters (2017, unknown): “The discussion here is related to Lee (2005), Bera et al. (2010), and Chernozhukov et al. (2015b).”
Double/de-biased machine learning using regularized Riesz representers (2018, background): “…[Zhang and Zhang (2014); Belloni et al. (2014a,b, 2015); Javanmard and Montanari (2014a,b, 2015); van de Geer et al. (2014); Ning and Liu (2014); Chernozhukov et al. (2015); Neykov et al. (2015); Ren et al. (2015); Jankova and Van De Geer (2015, 2016a,b); Bradic and Kolar (2017); Zhu and Bradic…”
Targeted Undersmoothing. (2017, background): “The ideas in the [13] can also be generalized to estimation of parameters defined by moment conditions whenever appropriate sparsity conditions hold and Neyman orthogonalizations of the moment conditions are available; see for example [26], [22], and references therein.”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, methodology): “…0 = o ( √ n/ log p ) ; see Zhang and Zhang (2014); Belloni et al. (2014); Van de Geer et al. (2014); Javanmard and Montanari (2014a); Ning and Liu (2014); Javanmard and Montanari (2015); Mitra and Zhang (2014); Bühlmann and van de Geer (2015); Belloni et al. (2015); Chernozhukov et al. (2015).”
Locally Robust Semiparametric Estimation (2016, methodology): “First, standard confidence intervals for debiased GMM are valid under local alternatives when ̂ uses model selection, as in Chernozhukov, Hansen, and Spindler (2015), but for plug-in GMM are typically biased and invalid, similar to Leeb and Potscher (2005).”
Taming the Factor Zoo: A Test of New Factors (2019, unknown): “…conditions and thereby reduce the dimensionality of ht, relying on this result produces very poor approximations to the finite-sample distributions of the estimators unless appropriate econometric methods are used to explicitly account for model selection mistakes (see Chernozhukov et al. (2015)).”