Nonparametric Scoring Rules

By Erik Zawadzki, Sébastien Lahaie. 2015

9 citations recorded by the source corpus.

A scoring rule is a device for eliciting and assessing probabilistic forecasts from an agent. When dealing with continuous outcome spaces, and absent any prior insights into the structure of the agent's beliefs, the rule should allow for a flexible reporting interface that can accurately represent complicated, multi-modal distributions. In this paper, we provide such a scoring rule based on a nonparametric approach of eliciting a set of samples from the agent and efficiently evaluating the score using kernel methods. We prove that sampled reports of increasing size converge rapidly to the true score, and that sampled reports are approximately optimal. We also demonstrate a connection between the scoring rule and the maximum mean discrepancy divergence. Experimental results are provided that confirm rapid convergence and that the expected score correlates well with standard notions of divergence, both important considerations for ensuring that agents are incentivized to report accurate information.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Economics and Econometrics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

DISCO Nets : DISsimilarity COefficients Networks (2016, background): “In our case, any loss ∆ expressed with an universal kernel will define the DISCO Nets’ objective function as such divergence (see Zawadzki and Lahaie [29]).”