Reliability diagrams for non‐parametric density forecasts of continuous variables: Accounting for serial correlation

By Pierre Pinson, Patrick E. McSharry, Henrik Madsen. 2010

83 citations recorded by the source corpus.

Reliability is seen as a primary requirement when verifying probabilistic forecasts, since a lack of reliability would introduce a systematic bias in subsequent decision‐making. Reliability diagrams comprise popular and practical diagnostic tools for the reliability evaluation of density forecasts of continuous variables. Such diagrams relate to the assessment of the unconditional calibration of probabilistic forecasts. A reason for their appeal is that deviations from perfect reliability can be visually assessed based on deviations from the diagonal. Deviations from the diagonal may, however, be caused by both sampling effects and serial correlation in the forecast‐verification pairs. We build on a recent proposal, consisting of associating reliability diagrams with consistency bars that would reflect the deviations from the diagonal that are potentially observable even if density forecasts are perfectly reliable. Our consistency bars, however, reflect potential deviations originating from the combined effects of limited counting statistics and serial correlation in the forecast‐verification pairs. They are generated based on an original surrogate consistency resampling method. Its ability to provide consistency bars with a significantly better coverage against the independent and identically distributed (i.i.d.) resampling alternative is shown from simulations. Finally, a practical example of the reliability assessment of non‐parametric density forecasts of short‐term wind‐power generation is given. Copyright © 2010 Royal Meteorological Society

Subfields: Atmospheric Science, Global and Planetary Change

View Paper: https://www.semanticscholar.org/paper/a19c410cf1ef16568fd7baf77f4faaaffd2acdd1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Global and Planetary Change

Citations in the corpus

An analog ensemble for short-term probabilistic solar power forecast (2015, methodology): “The vertical bars represent the confidence intervals [42,43] and are calculated with a quantile function for a binomial distribution.”
Sampling distributions of the Brier score and Brier skill score under serial dependence (2010, methodology): “…(2004) presented corrections to the chisquare statistic for evaluating uniformity of rank histograms and minimum spanning tree histograms, for autocorrelated forecast ensembles; and Pinson et al. (2010) consider the effect of serial correlation on the PIT (probability integral transform) histogram.”