Evaluating Probabilistic Forecasts with scoringRules

By Alexander I. Jordan, Fabian Krüger, Sebastian Lerch. Journal of Statistical Software, 2017

295 citations recorded by the source corpus.

Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields including meteorology, hydrology, economics, and demography. In typical applications, many alternative statistical models and data sources can be used to produce probabilistic forecasts. Hence, evaluating and selecting among competing methods is an important task. The scoringRules package for R provides functionality for comparative evaluation of probabilistic models based on proper scoring rules, covering a wide range of situations in applied work. This paper discusses implementation and usage details, presents case studies from meteorology and economics, and points to the relevant background literature.

Subfields: Artificial Intelligence, Atmospheric Science, Global and Planetary Change

View Paper: https://www.semanticscholar.org/paper/df52889bc4e34f9b7dc82ef2af3a00411b378177

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Atmospheric Science

In Global and Planetary Change

Citations in the corpus

Maximum likelihood estimation of spatially varying coefficient models for large data with an application to real estate price prediction (2020, methodology): “The values have been computed using the R package scoringRules (Jordan et al., 2019).”
Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting (2021, methodology): “F̂ (z) = 1 S PS s=1 I{x0,s  z} with S samples x0,s ⇠ F as a natural approximation of the predictive CDF, CRPS can be directly computed from simulated samples of the conditional distribution (8) at each time point (Jordan et al., 2019).”
Estimation Methods for Nonhomogeneous Regression Models: Minimum Continuous Ranked Probability Score versus Maximum Likelihood (2018, methodology): “…is performed in R (R Core Team 2017) using the crch package (Messner et al. 2016), which is able to perform minimization of the CRPS or LS. Closed expressions of the CRPS for the Gaussian, logistic, and Student’s t distribution are based on the scoringRules package (Jordan et al. 2017).”
Neural networks for post-processing ensemble weather forecasts (2018, methodology): “EMOS-gl and EMOS-loc are implemented in R with the help of the scoringRules package (Jordan et al., 2017).”