Estimating conditional quantiles with the help of the pinball loss

By Ingo Steinwart, Andreas Christmann. 2011

295 citations recorded by the source corpus.

Using the so-called pinball loss for estimating conditional quantiles is a well-known tool in both statistics and machine learning. So far, however, only little work has been done to quantify the efficiency of this tool for non-parametric (modified) empirical risk minimization approaches. The goal of this work is to fill this gap by establishing inequalities that describe how close approximate pinball risk minimizers are to the corresponding conditional quantile. These inequalities, which hold under mild assumptions on the data-generating distribution, are then used to establish so-called variance bounds which recently turned out to play an important role in the statistical analysis of (modified) empirical risk minimization approaches. To illustrate the use of the established inequalities, we then use them to establish an oracle inequality for support vector machines that use the pinball loss. Here, it turns out that we obtain learning rates which are optimal in a min-max sense under some standard assumptions on the regularity of the conditional quantile function.

Subfields: Computational Mechanics, Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e29474ca11604fb2edf68b560dc3a864610f5746

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, background): “(Quantile regression algorithms are not supported by ﬁnite sample coverage guarantees [32–36].)”
Optimal regression rates for SVMs using Gaussian kernels (2013, methodology): “For example, the articles [10, 11, 27, 5, 22, 32] and the references therein establish rates for SVMs using the least squares loss, while SVMs using the pinball loss are investigated in [29, 31].”
Conformalized Quantile Regression (2019, methodology): “ization problem q^ (x) = f(x;^); ^= argmin 1 n Xn i=1 ˆ (Y i;f(X i;))+R(); (5) where f(x;) is the quantile regression function and the loss function ˆ is the “check function” or “pinball loss” [18,24], deﬁned by ˆ (y;^y) := ˆ ( y ^) if y>0; (1 )(^y y) otherwise (6) and illustrated in Figure 1. T”
Separability of reproducing kernel spaces (2015, background): “7 and 12], [7], [6], Steinwart and Christmann [34], [35], De Vito, Rosasco and Toigo [9], Hable and Christmann [18, Thm.”
A Consistent Regularization Approach for Structured Prediction (2016, background): “Indeed, in the last few years, an effort has been made to analyze specific structured problems, such as multiclass classification [5], multi-labeling [6], ranking [7] or quantile estimation [8].”
The Big Data Newsvendor: Practical Insights from Machine Learning (2013, result): “The results in this paper thus also extend the literature on non-parametric quantile regression [see Koenker (2005) for a general reference on quantile regression, Takeuchi et al. (2006) and Steinwart and Christmann (2011) for up-to-date results at the time of writing].”
Single-Model Uncertainties for Deep Learning (2018, background): “The pinball loss τ estimates the τ -th quantile consistently [74], providing SQR with strong theoretical grounding.”