Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems

By Alexandre Belloni, Victor Chernozhukov, Kengo Kato. 2013

153 citations recorded by the source corpus.

We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against nonregular estimation of the nuisance part of the median regression function by using Neyman’s orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semiparametrically efficient. We also generalize our method to a general nonsmooth Z-estimation framework where the number of target parameters is possibly much larger than the sample size. We extend Huber's results on asymptotic normality to this setting, demonstrating uniform asymptotic normality of the proposed estimators over rectangles, constructing simultaneous confidence bands on all of the target parameters, and establishing asymptotic validity of the bands uniformly over underlying approximately sparse models.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, methodology): “Our paper also builds upon and contributes to the literature on modern orthogonal/debiased estimation and inference [Zhang and Zhang (2014); Belloni et al. (2011, 2014, 2015); Javanmard and Montanari (2014a,b, 2018); Van de Geer et al. (2014); Ning and Liu (2017); Chernozhukov et al. (2015); Neykov…”
Penalized expectile regression: an alternative to penalized quantile regression (2018, background): “…p is much larger than n. Belloni and Chernozhukov (2011) study theory of quantile regression with lasso penalty in an ultra-high-dimensional setting, and Belloni et al. (2015) develop post-selection inference methods for quantile regression with lasso penalty in an ultra-high- dimensional setting.”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, methodology): “…in this domain are, however, restricted to the ultra-sparse case, where (cid:107) β ∗ (cid:107) 0 = o ( √ n/ log p ) ; see Zhang and Zhang (2014); Belloni et al. (2014); Van de Geer et al. (2014); Javanmard and Montanari (2014a); Ning and Liu (2014); Javanmard and Montanari (2015); Mitra…”
Bootstrapping and sample splitting for high-dimensional, assumption-lean inference (2016, background): “…2009); Meinshausen (2015) on inference about groups of variables under general designs; Belloni et al. (2011) in the instrumental variable setting; Belloni et al. (2015) on post-selection inference for Z-estimators, and the knockoffs approach of Barber et al. (2015) and later Candes et al.…”
Asymptotic post‐selection inference for the Akaike information criterion (2018, methodology): “See also Belloni et al. (2015) for the use of least absolute deviation in high-dimensional regression.”
Orthogonal Machine Learning for Demand Estimation: High Dimensional Causal Inference in Dynamic Panels (2017, methodology): “is partialling out approach has classical roots in econometrics, going back at least to Frisch and Waugh. In conjunction with machine learning, it was used in high-dimensional sparse linear models in Belloni et al. (2014) and with generic machine learning methods in Chernozhukov et al. (2016); in th”
High-dimensional econometrics and regularized GMM (2018, methodology): “Similar results formulated in terms of particular applications can be found for example in [19], [11], and [17].”
Beyond Gaussian approximation: Bootstrap for maxima of sums of independent random vectors (2017, methodology): “It can be also used in time series settings (Zhang and Wu, 2017a) and high-dimensional regression (Zhang and Zhang, 2014; Belloni et al., 2014, 2015; Zhang and Cheng, 2017; Dezeure et al., 2017).”
On LASSO for predictive regression (2018, unknown): “Remark 3.6 Another related results in the literature are uniformly valid inference and forecasting after the LASSO model selection, see Belloni et al. (2015, 2018) or Hirano and Wright (2017), for example.”
Semiparametric theory and empirical processes in causal inference (2015, background): “additive models, and partially linear models (see for example [18] for a review of such models, which typically yield estimators with n−2/5 rates), as well as regularized estimators such as the Lasso [5, 6]. Cross-validation-based weighted combinations of such estimators (e.g., Super Learner) can al”
Double/debiased machine learning for treatment and structural parameters (2017, unknown): “7The C(α)-statistic, or the orthogonal score statistic, has been explicitly used for testing and estimation in high-dimensional sparse models in Belloni et al. (2015).”
Improved central limit theorem and bootstrap approximations in high dimensions (2019, unknown): “[5]), the statistic Tn can only be asymptotically approximated by the maximum coordinate of the sample mean of independent random vectors.”
Estimation and Inference about Heterogeneous Treatment Effects in High-Dimensional Dynamic Panels. (2017, background): “The first major novelty of this analysis is to show that the standard choice (3.6) also dominates the first-stage estimation error, a type of error that has not arised in prior Lasso estimators (e.g. Belloni et al. (2014b)).”