Proper local scoring rules

By Matthew Parry, A. P. Dawid, Steffen L. Lauritzen. 2011

149 citations recorded by the source corpus.

We investigate proper scoring rules for continuous distributions on the real line. It is known that the log score is the only such rule that depends on the quoted density only through its value at the outcome that materializes. Here we allow further dependence on a finite number $m$ of derivatives of the density at the outcome, and describe a large class of such $m$-local proper scoring rules: these exist for all even $m$ but no odd $m$. We further show that for $m\geq2$ all such $m$-local rules can be computed without knowledge of the normalizing constant of the distribution.

Subfields: Economics and Econometrics, Management Science and Operations Research, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015, background): “…instance mean ﬁeld theory and its expansions [39, 40], variational Bayes [19], contrastive divergence [44, 13], minimum probability ﬂow [36, 35], minimum KL contraction [28], proper scoring rules [9, 32], score matching [15], pseudolikelihood [5], loopy belief propagation [29], and many, many more.”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “1, the solution (13) is not only optimal for the minimax Brier risk, but is in fact optimal for a wide family of strictly proper scoring rules known as the (separable) Bregman score (Parry et al., 2012):”
Strictly Proper Scoring Rules (2014, background): “The notion of locality in infinite continuous sample spaces has been extended to allow the loss function to also depend on the derivatives of bel(X), see [41].”
Using Stacking to Average Bayesian Predictive Distributions (with Discussion) (2017, background): “See Dawid (1986); Dawid and Sebastiani (1999); Grünwald and Dawid (2004); Parry et al. (2012); Dawid and Musio (2014, 2015); Dawid et al. (2016) for a variety of theory, examples and applications.”
Does data splitting improve prediction? (2013, background): “It is a proper scoring rule as defined by [25].”
Minimum Scoring Rule Inference (2014, background): “For the case that X is a real interval, Parry et al. (2012) show that any such local proper scoring rule is a linear combination of the log score and what they term a key local scoring rule, which they have characterised.”
How to Train Your Energy-Based Models (2021, background): “It is also possible to consider higher-order gradients of log-PDFs beyond ﬁrst derivatives (Parry et al., 2012).”
Trainable Calibration Measures For Neural Networks From Kernel Mean Embeddings (2018, background): “This implies that MMCE is a proper scoring rule (Parry et al., 2011).”
Model Selection Techniques: An Overview (2018, background): “There can be a number of variations to this in terms of the prediction loss function [8] and time dependency.”