Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings

By W. Ehm, Tilmann Gneiting, Alexander I. Jordan, Fabian Krüger. 2015

185 citations recorded by the source corpus.

In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional. For example, forecasters might be asked to report the mean or a quantile of their predictive distributions. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be consistent for the functional at hand, in the sense that the expected score is minimized when following the directive. We show that any scoring function that is consistent for a quantile or an expectile functional can be represented as a mixture of elementary or extremal scoring functions that form a linearly parameterized family. Scoring functions for the mean value and probability forecasts of binary events constitute important examples. The extremal scoring functions admit appealing economic interpretations of quantiles and expectiles in the context of betting and investment problems. The Choquet‐type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function. In empirical settings it suffices to compare the average scores for only a finite number of extremal elements. Plots of the average scores with respect to the extremal scoring functions, which we call Murphy diagrams, permit detailed comparisons of the relative merits of competing forecasts.

Subfields: Economics and Econometrics, Finance, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/36e16161295762caa8d9259ec3fc574fd2e87012

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Finance

In Management Science and Operations Research

Citations in the corpus

Estimation of tail risk based on extreme expectiles (2016, unknown): “Also, Ehm et al. (2016) have shown that expectiles are optimal decision thresholds in certain binary investment problems.”
Comparing Possibly Misspecified Forecasts (2020, methodology): “In recent work, Ehm et al. (2016) obtained mixture representations for the classes of loss functions consistent for quantiles and expectiles which can be used to determine whether one forecast outperforms another across all consistent loss functions.”
A joint quantile and expected shortfall regression framework (2017, unknown): “We use Murphy diagrams introduced by Ehm et al. (2016) and Ziegel et al. (2017), which provide a parsimonious way to evaluate competing forecasts simultaneously for a full class of strictly consistent loss functions.”
Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination (2021, methodology): “In all cases studied thus far, canonical loss functions are strictly consistent (Ehm et al., 2016), and so MCBu = 0 if and only if the forecast is unconditionally T-calibrated, and MCBc = 0 if and only if Xurc = Xrc almost surely.”