A closed-form expression for the Sharma–Mittal entropy of exponential families
By Frank Nielsen, Richard Nock. 2011
107 citations recorded by the source corpus.
The Sharma–Mittal entropies generalize the celebrated Shannon, Rényi and Tsallis entropies. We report a closed-form formula for the Sharma–Mittal entropies and relative entropies for arbitrary exponential family distributions. We explicitly instantiate the formula for the case of the multivariate Gaussian distributions and discuss its estimation.
Subfields: Statistical and Nonlinear Physics, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistical and Nonlinear Physics
- Arthur Gretton
- Bharath K. Sriperumbudur
- Zoltán Szabó
- Barnab'as P'oczos
- Peter S. Dayan
- T. Griffiths
- David A. Lagnado
- Neil R. Bramley
In Statistics and Probability
- Bharath K. Sriperumbudur
- Arthur Gretton
- Bernhard Schölkopf
- Zoltán Szabó
- Kenji Fukumizu
- Barnab'as P'oczos
- Krikamol Muandet
- Dino Sejdinovic
Citations in the corpus
- Learning Theory for Distribution Regression (2014, background): “Typical examples with explicit formulas include Gaussians, finite mixtures of Gaussians, and distributions from the exponential family (with known log-normalizer function and zero carrier measure, see Kondor and Jebara, 2003; Jebara et al., 2004; Wang et al., 2009; Nielsen and Nock, 2012).”
- Formalizing Neurath’s Ship: Approximate Algorithms for Online Causal Learning (2016, background): “Shannon is just one of a broad family of possible entropy measures (Nielsen & Nock, 2011).”
- On Measures of Entropy and Information (2015, methodology): “Tsallis information [28–30] (Havrda-Charvat information, (cid:11) order information) Sharma-Mittal information [31, 32] Assuming suitable limits are taken, the Sharma-Mittal information contains Shannon, R´enyi and Tsallis informations as special cases.”