Information, Divergence and Risk for Binary Experiments

By Mark D. Reid, Robert C. Williamson. 2009

250 citations recorded by the source corpus.

We unify f-divergences, Bregman divergences, surrogate regret bounds, proper scoring rules, cost curves, ROC-curves and statistical information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their representation primitives which all are related to cost-sensitive binary classification. As well as developing relationships between generative and discriminative views of learning, the new machinery leads to tight and more general surrogate regret bounds and generalised Pinsker inequalities relating f-divergences to variational divergence. The new viewpoint also illuminates existing algorithms: it provides a new derivation of Support Vector Machines in terms of divergences and relates maximum mean discrepancy to Fisher linear discriminants.

Subfields: Artificial Intelligence, Statistical Learning Theory, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Bernhard Schölkopf
  2. Yoshua Bengio
  3. Dacheng Tao
  4. Sergey Levine
  5. Arthur Gretton
  6. Alex Smola
  7. Ruslan Salakhutdinov
  8. Quoc V. Le

In Statistical Learning Theory

  1. Arthur Gretton
  2. Bernhard Schölkopf
  3. Kenji Fukumizu
  4. Bharath K. Sriperumbudur
  5. Alex Smola
  6. Karsten M. Borgwardt
  7. Le Song
  8. Balaji Lakshminarayanan

In Information Theory

  1. Balaji Lakshminarayanan
  2. Shivani Agarwal
  3. Igal Sason
  4. Shakir Mohamed
  5. Marco Loog
  6. Clayton D. Scott
  7. Gavin E. Crooks
  8. Bernhard Schölkopf

Citations in the corpus