Estimation of the Continuous Ranked Probability Score with Limited Information and Applications to Ensemble Weather Forecasts
By M. Zamo, Philippe Naveau. Mathematical Geosciences, 2018
175 citations recorded by the source corpus.
The continuous ranked probability score (CRPS) is a much used measure of performance for probabilistic forecasts of a scalar observation. It is a quadratic measure of the difference between the forecast cumulative distribution function (CDF) and the empirical CDF of the observation. Analytic formulations of the CRPS can be derived for most classical parametric distributions, and be used to assess the efficiency of different CRPS estimators. When the true forecast CDF is not fully known, but represented as an ensemble of values, the CRPS is estimated with some error. Thus, using the CRPS to compare parametric probabilistic forecasts with ensemble forecasts may be misleading due to the unknown error of the estimated CRPS for the ensemble. With simulated data, the impact of the type of the verified ensemble (a random sample or a set of quantiles) on the CRPS estimation is studied. Based on these simulations, recommendations are issued to choose the most accurate CRPS estimator according to the type of ensemble. The interest of these recommendations is illustrated with real ensemble weather forecasts. Also, relationships between several estimators of the CRPS are demonstrated and used to explain the differences of accuracy between the estimators.
Subfields: Atmospheric Science, Global and Planetary Change
View Paper: https://www.semanticscholar.org/paper/09839ff31da3cdeadd8f27156eb13e7a24ccb95f
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Atmospheric Science
- Stephan Rasp
- Alberto Arribas
- Shakir Mohamed
- Martin Leutbecher
- Stephan Hoyer
- Jonathan A. Weyn
- I. Pisso
In Global and Planetary Change
Citations in the corpus
- Ensemble size: How suboptimal is less than infinity? (2018, background): “…keeping in mind, however, that it may be not fair when members are not exchangeable or when the sampled distribution changes with ensemble size M. Zamo and Naveau (2018) show that the fair CRPS is equivalent to the probability-weighted moment form of the CRPS and discuss the properties of…”
- Skilful precipitation nowcasting using deep generative models of radar (2021, methodology): “We compute unbiased estimates of CRPS using CRPSPWM from [41], with the N ensemble members as samples.”
- WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models (2023, background): “In the case of a deterministic prediction, the CRPS reduces to the MAE.”