L1-Penalised quantile regression in high-dimensional sparse models

By Alexandre Belloni, Victor Chernozhukov. 2009

435 citations recorded by the source corpus.

We consider median regression and, more generally, a possibly infinite collection of quantile regressions in high-dimensional sparse models. In these models the overall number of regressors $p$ is very large, possibly larger than the sample size $n$, but only $s$ of these regressors have non-zero impact on the conditional quantile of the response variable, where $s$ grows slower than $n$. We consider quantile regression penalized by the $\ell_1$-norm of coefficients ($\ell_1$-QR). First, we show that $\ell_1$-QR is consistent at the rate $\sqrt{s/n} \sqrt{\log p}$. The overall number of regressors $p$ affects the rate only through the $\log p$ factor, thus allowing nearly exponential growth in the number of zero-impact regressors. The rate result holds under relatively weak conditions, requiring that $s/n$ converges to zero at a super-logarithmic speed and that regularization parameter satisfies certain theoretical constraints. Second, we propose a pivotal, data-driven choice of the regularization parameter and show that it satisfies these theoretical constraints. Third, we show that $\ell_1$-QR correctly selects the true minimal model as a valid submodel, when the non-zero coefficients of the true model are well separated from zero. We also show that the number of non-zero coefficients in $\ell_1$-QR is of same stochastic order as $s$. Fourth, we analyze the rate of convergence of a two-step estimator that applies ordinary quantile regression to the selected model. Fifth, we evaluate the performance of $\ell_1$-QR in a Monte-Carlo experiment, and illustrate its use on an international economic growth application.

Subfields: Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b98054f4a8582af49c77e53fcd9e68328ce7c01e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Sparse Quantile Huber Regression for Efficient and Robust Estimation (2014, methodology): “Therefore, we extend Belloni & Chernozhukov [4] to consider greedy and convex formulations for sparse quantile regression in a high-dimensional setting.”
Penalized expectile regression: an alternative to penalized quantile regression (2018, background): “Wang et al. (2012) study the behavior of quantile regression in ultra-high-dimensional data when p is much larger than n. Belloni and Chernozhukov (2011) study theory of quantile regression with lasso penalty in an ultra-high-dimensional setting, and Belloni et al. (2015) develop post-selection…”
An Iterative Coordinate Descent Algorithm for High-Dimensional Nonconvex Penalized Quantile Regression (2015, methodology): “We refer to the recent survey papers by Bickel and Li (2006) and Fan and Lv (2010) for many relevant references.”
Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain (2012, methodology): “Appendix E. Proofs of Lemmas 1-4 For proof of Lemma 1, see Belloni and Chernozhukov (2011a), Supplement.”
Program evaluation and causal inference with high-dimensional data (2013, methodology): “The proof of the following lower bound builds upon ideas developed in Belloni and Chernozhukov (2011) for high-dimensional quantile regressions.”
Penalized Quantile Regression for Dynamic Panel Data (2010, background): “1 See Hsiao (2003), Arellano (2003) and Heckman (1981) for more details.”
Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models (2013, methodology): “In particular, we establish new finite sample bounds for the prediction norm on the estimation error of 1-penalized quantile regression in nonparametric models that extend results of Belloni and Chernozhukov (2011) and Kato (2011).”
Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems (2013, background): “Indeed, the estimators (2) and (3) attain essentially optimal rates {(s log p)/n}1/2 of convergence for estimating the entire median regression function [3, 33].”
Adaptive Huber Regression (2017, methodology): “…main focus is on the conditional mean regression in the presence of heavy-tailed and asymmetric errors, which automatically distinguishes our method from quantile-based robust regressions (Koenker 2005; Belloni and Chernozhukov 2011; Wang 2013; Fan, Fan, and Barut 2014; Zheng, Peng, and He 2015).”
Group Lasso for high dimensional sparse quantile regression models (2011, methodology): “For instance, it is true for the case when x 1 , − 1 has a log concave density (Belloni and Chernozhukov, 2011, Comment 2.2).”
TENET: Tail-Event driven NETwork risk (2016, methodology): “(2003), Li and Zhu (2008), Belloni and Chernozhukov (2011), etc.”
A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression (2014, methodology): “In high-dimensional settings, Belloni and Chernozhukov (2011), Wang (2013), and Wang et al. (2012) established point estimation theories for the penalized least absolute deviation and quantile regressions.”