Theory and applications of proper scoring rules

By A. P. Dawid, Monica Musio. 2014

141 citations recorded by the source corpus.

A scoring rule $$S(x; q)$$S(x;q) provides a way of judging the quality of a quoted probability density $$q$$q for a random variable $$X$$X in the light of its outcome $$x$$x. It is called proper if honesty is your best policy, i.e., when you believe $$X$$X has density $$p$$p, your expected score is optimised by the choice $$q = p$$q=p. The most celebrated proper scoring rule is the logarithmic score, $$S(x; q) = -\log {q(x)}$$S(x;q)=-logq(x): this is the only proper scoring rule that is local, in the sense of depending on the density function $$q$$q only through its value at the observed value $$x$$x. It is closely connected with likelihood inference, with communication theory, and with minimum description length model selection. However, every statistical decision problem induces a proper scoring rule, so there is a very wide variety of these. Many of them have additional interesting structure and properties. At a theoretical level, any proper scoring rule can be used as a foundational basis for the theory of subjective probability. At an applied level a proper scoring can be used to compare and improve probability forecasts, and, in a parametric setting, as an alternative tool for inference. In this article we give an overview of some uses of proper scoring rules in statistical inference, including frequentist estimation theory and Bayesian model selection with improper priors.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/2c67bb06048083547135519b22bc2e68d454ac37

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Using Stacking to Average Bayesian Predictive Distributions (with Discussion) (2017, background): “See Dawid (1986); Dawid and Sebastiani (1999); Grünwald and Dawid (2004); Parry et al. (2012); Dawid and Musio (2014, 2015); Dawid et al. (2016) for a variety of theory, examples and applications.”
NGBoost: Natural Gradient Boosting for Probabilistic Prediction (2019, background): “The excess score of the right hand side over the left is the divergence induced by that scoring rule (Dawid and Musio, 2014):”
Minimum Scoring Rule Inference (2014, methodology): “Both full and pseudo likelihood inference are special cases of a more general estimation technique based on proper scoring rules (see, e.g., Dawid and Musio, 2014), which are methods for encouraging honest assessment of probability distributions.”
Generative Adversarial Nets from a Density Ratio Estimation Perspective (2016, unknown): “For details, see [36, 35, 7, 26].”