Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size

By D. Richardson. 2001

213 citations recorded by the source corpus.

Subfields: Atmospheric Science, Global and Planetary Change

View Paper: https://www.semanticscholar.org/paper/33934b12f41be99f491db074f2a7e5753dfb1773

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Global and Planetary Change

Citations in the corpus

Comparing Probabilistic Forecasting Systems with the Brier Score (2007, methodology): “Richardson (2001) does, however, develop a method for estimating a skill score based on BM given an ensemble of any size, even m 1.”
Skill of Multimodel ENSO Probability Forecasts (2008, background): “Generalizing the results of Richardson (2001) and Tippett et al. (2007), the RPSS of a perfectly reliable forecast system depends on ensemble size N according to (see appendix B for details): Setting (cid:9) RPSS(9) (cid:10) (cid:1) 0.5, gives (cid:9) RPSS( (cid:12) ) (cid:10) (cid:1) 0.55, and…”
Benefits of increased resolution in the ECMWF ensemble system and comparison with poor‐man's ensembles (2003, methodology): “…forecast can be considered as a single representative member of such an ideal ensemble, and the BSS of the control, BSSC, can be used to estimate the BSS of this hypothetical perfect ensemble and of a nite sized, M-member ensemble drawn from this perfect distribution (Richardson 2001): BSSperfM D .”
Estimation of Seasonal Precipitation Tercile-Based Categorical Probabilities from Ensembles (2007, methodology): “We note that a simpler calibration method based on a Gaussian fit with the variance determined by the correlation between ensemble mean and observations, as in Tippett et al. (2005), rather than ensemble spread performs nearly as well as the Gaussian fit with Bayesian calibration.”
Comparison of ensemble‐MOS methods in the Lorenz '96 setting (2006, background): “…Pr(V ≤ q) = 1 when all ensemble members are smaller than q, and Pr(V ≤ q) = 0 when all ensemble members are larger than q. Note, however, that Equation (1) does not fully correct the conditional biases in probability estimation from ensemble relative frequencies discussed by Richardson (2001).”
The discrete brier and ranked probability skill scores (2007, background): “Indeed, the RPSS is negatively biased (Richardson 2001; Kumar et al. 2001; Mason 2004).”
Ensemble forecasting (2008, background): “Real ensembles with a finite number of members cannot be perfectly reliable due to sampling uncertainty [20].”
Probabilistic Seasonal Forecasts in the North American Multimodel Ensemble: A Baseline Skill Assessment (2016, result): “While the beneficial effect of larger ensemble sizes is well known (e.g., Déqué 1997; Kumar et al. 2001; Richardson 2001), the results of this study suggest that the higher skill seen in the NMME is not merely the effect of a larger ensemble.”
Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation (2005, background): “Hence, raw ensemble output does not provide predictive PDFs, and some form of postprocessing is required (Hamill and Colucci 1998; Richardson 2001).”
The rationale behind the success of multi-model ensembles in seasonal forecasting — II. Calibration and combination (2005, background): “A skill score can be constructed from this area as ROCSS = 2 × A − 1 (Richardson, 2001).”
The economic value of ensemble forecasts as a tool for risk assessment: From days to decades (2002, methodology): “Over the years, Angström’s model has been re ned and developed by Thompson (1952), Murphy (1977), Katz and Murphy (1997) and Richardson (2000a, 2001).”
The Multiensemble Approach: The NAEFS Example (2009, methodology): “…from 20 to 40 members. b. Impact of the ensemble size N Several studies have already shown the impact of the ensemble size N on the global Brier score (Talagrand Unauthenticated | Downloaded 08/17/21 11:09 AM UTC et al. 1999; Richardson 2001b): the measure converges to its theoretical limit as 1/N.”
Skill and relative economic value of medium-range hydrological ensemble predictions (2006, background): “The ensembles are taken at “face value” (e.g. Richardson 2001; Katz and Ehrendorfer 2006) and the uncertainty due to the finite size of the ensembles is ignored.”
Performance targets and the Brier score (2007, background): “Of course the forecaster could be a ‘she’, although given the trend towards automated forecast products the forecaster is increasingly likely to be an ‘it’. 2. There is a different version of the linear score which is proper (Bröcker and Smith, 2006). 3. In their article on the effect of non-linear ”
Ensemble size: How suboptimal is less than infinity? (2018, background): “For the Brier score, Richardson (2001) showed that, for perfect reliability, the score of an M-member ensemble relates to the score of an infinite-size ensemble as BSM = (1 + M−1)BS∞, (19) see his equations 8 and 9.”
Improvement of ensemble reliability with a new dressing kernel (2005, background): “Thus the improvement of the dressed-ensemble scores relative to the undressed-ensemble scores may partly come from the increase of the ensemble size (Richardson 2001; RS03).”
Ensemble flood forecasting: a review. (2009, background): “In the longer term, forecast centres should plan to increase the resolution of their forecasts and the number of ensemble members in their EPS to represent covariance of predictors and the extremes of distributions adequately (for a discussion please see Doblas-Reyes et al., 2008; Ferro et al., 2008”