Axiomatic Characterizations of Information Measures

By Imre Csiszár. Entropy, 2008

310 citations recorded by the source corpus.

Axiomatic characterizations of Shannon entropy, Kullback I-divergence, and some generalized information measures are surveyed. Three directions are treated: (A) Characterization of functions of probability distributions suitable as information measures. (B) Characterization of set functions on the subsets of {1; : : : ;N} representable by joint entropies of components of an N-dimensional random vector. (C) Axiomatic characterization of MaxEnt and related inference rules. The paper concludes with a brief discussion of the relevance of the axiomatic approach for information theory.

Subfields: Cognitive Neuroscience, Computational Theory and Mathematics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/c6aac5dc1c267e2f0dccab9197fe2e9f72ee792c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computational Theory and Mathematics

In Statistical and Nonlinear Physics

Citations in the corpus

On Rényi measures and hypothesis testing (2011, background): “Besides their operational importance, these quantities also possess some very natural properties one would expect an information measure to satisfy, a fact that has spurred several different axiomatic characterizations, see [1] and references therein.”
Probabilism, entropies and strictly proper scoring rules (2015, background): “613 MaxEnt has given rise to a substantial literature on rational belief for614 mation; as examples we mention [1, 8, 23, 29, 40, 41].”
State of the field: Measuring information and confirmation (2014, background): “Various axiomatic approaches have been pursued in the definition of entropymeasures (see Chakrabarti & Ghosh, 2009; Csizár, 2008 for recent overviews and further references).”
Thermodynamics as a theory of decision-making with information-processing costs (2012, background): “The logarithmic mapping between probabilities and “costs” is wellknown in information theory, and there are many possible ways to derive it [20, 21].”
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods (2019, background): “…a measure of total uncertainty as well as its decomposition into aleatoric and epistemic parts from basic principles in a mathematically rigorous way, i.e., to develop a kind of axiomatic basis for such a decomposition, comparable to axiomatic justifications of the entropy measure (Csiszár, 2008).”
Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities (2010, background): “The Csiszár–Morimoto f -divergence has many beautiful properties [6,17,18,20,22,36,52,53]:”