Composite quantile regression and the oracle Model Selection Theory

By Hui Zou, Ming Yuan. 2008

614 citations recorded by the source corpus.

Coefficient estimation and variable selection in multiple linear regression is routinely done in the (penalized) least squares (LS) framework. The concept of model selection oracle introduced by Fan and Li [J. Amer. Statist. Assoc. 96 (2001) 1348--1360] characterizes the optimal behavior of a model selection procedure. However, the least-squares oracle theory breaks down if the error variance is infinite. In the current paper we propose a new regression method called composite quantile regression (CQR). We show that the oracle model selection theory using the CQR oracle works beautifully even when the error variance is infinite. We develop a new oracular procedure to achieve the optimal properties of the CQR oracle. When the error variance is finite, CQR still enjoys great advantages in terms of estimation efficiency. We show that the relative efficiency of CQR compared to the least squares is greater than 70% regardless the error distribution. Moreover, CQR could be much more efficient and sometimes arbitrarily more efficient than the least squares. The same conclusions hold when comparing a CQR-oracular estimator with a LS-oracular estimator.

Subfields: Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c6ceb130f0010c82abcf97f4aa988642a67c903d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, background): “Zou and Yuan (2008) propose the penalized composite quantile with equal weights to improve the eﬃciency of the penalized least squares.”
Robust rank correlation based screening (2010, methodology): “such screening method should be expected to be more robust than the SIS proposed by Fan and Lv (2008). Following such robust sure independent screening, most robust variable selection methods such as Zou and Yuan (2008) and Li, Peng and Zhu (2011) can be used to get a robust reﬁned regression model ”
Estimation of High Conditional Quantiles for Heavy-Tailed Distributions (2012, background): “This idea appeared in Koenker ( 1984 ), Koenker ( 2004 ) and Zou and Yuan ( 2008 ).”
Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores (2017, background): “In the context of quantile regression, a lot of the early work focused on estimation, prediction and variable selection consistency in low (Li and Zhu, 2008; Zou and Yuan, 2008; Wu and Liu, 2009) and high-dimensional setting (Belloni and Chernozhukov, 2011; Wang et al., 2012; Zheng et al., 2013).”
Posterior Inference in Bayesian Quantile Regression with Asymmetric Laplace Likelihood (2016, background): “International Statistical Review © 2015 International Statistical Institute. objective function over multiple quantiles, such as the objective function in the composite quantile regression of Zou & Yuan (2008).”
Adaptive penalized quantile regression for high dimensional data (2013, methodology): “Exploiting the ideas of Wang et al. (2007) and Zou and Yuan (2006), we use the consistent estimator from Belloni and Chernozhukov (2011) to determine adaptive weights.”
Loss Minimization and Parameter Estimation with Heavy Tails (2013, background): “…heavy-tailed noise under quantile assumptions, for instance when the median of the noise is zero ( e.g. , Wang 2013; Belloni and Chernozhukov 2011; Zou and Yuan 2008; Wu and Liu 2009; Wang et al. 2007; Fan et al. 2012). van de Geer and M¨uller (2012) analyze a class of optimization functions that…”
Asymmetric Laplace Regression: Maximum Likelihood, Maximum Entropy and Quantile Regression (2015, background): “In a related context, Zou and Yuan (2008) develop the composite quantile regression (CQR) estimator where the slope parameters of interest are obtained from a mixture of the objective functions from different quantile regression models.”
Bayesian empirical likelihood for quantile regression (2012, methodology): “The three BEL methods (BEL.s, BEL.c and BEL.n) will be compared with RQ and the composite quantile regression (CQR) of Zou and Yuan (2008).”
SPARSE VARYING COEFFICIENT MODELS FOR LONGITUDINAL DATA (2010, background): “Fan and Li (2001), Zou (2006), and Zou and Yuan (2008), among others, established this property in parametric linear models.”
STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION (2012, background): “Both δ 1 and δ 2 are related to the regularity behavior of the oracle estimator and will be referred to the oracle regularity condition.”
High‐dimensional quantile regression: Convolution smoothing and concave regularization (2021, result): “The results of this section complement the existing theory for composite quantile regression in high dimensions (Zou and Yuan, 2008; Bradic, Fan and Wang, 2011).”
Tail Index Regression (2009, neutral): “Specifically, the RE values are always larger than 74% and additionally above 85% for large sample sizes, which seems acceptable in practice; see, for example, Zou and Yuan (2008). Hence, the sample fraction determination suggested in Section 2.”
Composite Quantile Regression for the Single-Index Model (2013, neutral): “2012/08/31 file: 20130205*Hae*Wan*Zhu*Composite*Quantile*Regression*SIM.tex date: February 6, 2013 inference.”
Composite quantile regression neural network with applications (2017, positive): “It indicates that K = 19 is a better choice, which is in line ith Zou and Yuan (2008) .”
An Iterative Coordinate Descent Algorithm for High-Dimensional Nonconvex Penalized Quantile Regression (2015, background): “It is also noteworthy that in the fixed p case, the theory for penalized quantile regression was investigated by Zou and Yuan (2008), Wu and Liu (2009), Kai, Li, and Zou (2011), among others.”