Scoring Rules for Continuous Probability Distributions

By J. Matheson, Robert L. Winkler. 1976

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes (2019, methodology): “(a) Effect of the number of evaluation samples ( num eval samples ) on the CRPS for electricity, solar, and taxi datasets.”
Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds (2008, methodology): “This scoring rule can be traced back to Matheson and Winkler (1976) and is increasingly being used in the meteorological community (p. 252, Hersbach 2000; Candille and Talagrand 2005; Wilks 2006).”
Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules (2011, methodology): “In this setting, the optimal point forecast or Bayes rule is the α-quantile, F−1(α), of the predictive distribution, F (Granger 1969; Matheson and Winkler 1976), and the quantile score, QSα(F −1(α), y), equals the corresponding loss.”
A penny for your thoughts: a survey of methods for eliciting beliefs (2014, unknown): “Matheson and Winkler (1976) shows that the quadratic, spherical and logarithmic scoring rules can be modiﬁed to generate proper scoring rules for density function elicitation.”
Performance and reliability of multimodel hydrological ensemble simulations based on seventeen lumped models and a thousand catchments (2010, unknown): “The selected score is the Continuous Ranked Probability Score (CRPS) (Matheson and Winkler, 1976), which is a proper score widely used in atmospheric and hydrologic sciences (e.g., Gneiting et al., 2005; Candille and Talagrand, 2005; Weber et al., 2006; Boucher et al., 2009).”
Reliability, sufficiency, and the decomposition of proper scores (2009, background): “πγ = γ. Since πγ is reliable, it trivially holds that πγ = π(πγ). In any case, πγ is a function of γ, independent of whether γ is reliable or not. Let usturn ourattention toscoring rules(seeforexampleMatheson and Winkler, 1976; Gneiting and Raftery, 2007). A scoring rule is a function S(p,y) which t”
Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion (2024, unknown): “We didn’t observe the robot act wildly due to visual distractors.”
SEAS5: the new ECMWF seasonal forecast system (2018, methodology): “It is common to use CRPSS, the skill score version of the continuous ranked probability score (CRPS; Hersbach, 2000; Matheson and Winkler, 1976; Wilks, 2011), to evaluate the”
Geostatistical Modelling Using Non‐Gaussian Matérn Fields (2015, background): “We displays four statistics from the cross-validation predictions: the variance of the residuals; the variance of the standardized residuals; the continuous ranked probability score (CRPS) (Matheson & Winkler, 1976), which probably is the most employed scoring role in probabilistic forecasts; and th”
Combining Predictive Distributions (2011, methodology): “…distributions are also absolutely continuous, and our preferred estimation technique is to maximize the mean (or sum) of the logarithmic score (Matheson and Winkler 1976; Gneiting and Raftery 2007) over the training data, namely ℓ(w1, . . . , wk;α, β) = J∑ j=1 log(gα,β(yj)) (10) =…”
Comparison of ensemble‐MOS methods in the Lorenz '96 setting (2006, background): “…et al. (2005) also propose the innovation of choosing the parameters to minimise the average continuous ranked probability score (Hersbach 2000; Matheson & Winkler 1976; Wilks 2006) for the training data, CRP S = 1 n n∑ i=1 ( c + d σ̂ 2ens, i )1/2( zi[2 (zi) − 1] + 2φ(zi) − π−1/2 ) ,…”
Probabilistic quantitative precipitation field forecasting using a two-stage spatial model (2008, methodology): “To assess the quality of the predictive distributions for daily precipitation accumulation, we use the continuous ranked probability score [Matheson and Winkler (1976), Gneiting and Raftery (2007)], which is a strictly proper scoring rule for the evaluation of probabilistic forecasts of a…”