On the Comparison of Interval Forecasts

By Ross Askanazi, Francis X. Diebold, Frank Schorfheide, Minchul Shin. Journal of Time Series Analysis, 2018

47 citations recorded by the source corpus.

We explore interval forecast comparison when the nominal confidence level is specified, but the quantiles on which intervals are based are not specified. It turns out that the problem is difficult, and perhaps unsolvable. We first consider a situation where intervals meet the Christoffersen conditions (in particular, where they are correctly calibrated), in which case the common prescription, which we rationalize and explore, is to prefer the interval of shortest length. We then allow for mis‐calibrated intervals, in which case there is a calibration‐length tradeoff. We propose two natural conditions that interval forecast loss functions should meet in such environments, and we show that a variety of popular approaches to interval forecast comparison fail them. Our negative results strengthen the case for abandoning interval forecasts in favor of density forecasts: Density forecasts not only provide richer information, but also can be readily compared using known proper scoring rules like the log predictive score, whereas interval forecasts cannot.

Subfields: Finance, Global and Planetary Change, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/d5482d8f608c91cc78724a54abb537787c923319

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Finance

In Global and Planetary Change

In Management Science and Operations Research

Citations in the corpus

Forecasting in social settings: The state of the art (2020, background): “For a discussion of some of the problems with interval scoring, see Askanazi et al. (2018).”
Forecasting: theory and practice (2020, background): “ch require more thought in choosing among than is obvious at ﬁrst glance. The diﬃculties involved in evaluating interval and pHDR forecasts motivate a stronger emphasis on full density forecasts (cf. Askanazi et al., 2018). 2.11.2. Evaluating probabilistic forecasts76 Probabilistic forecasting is a ”