Some inequalities for information divergence and related measures of discrimination

By Flemming Topsøe. IEEE Transactions on Information Theory, 2000

348 citations recorded by the source corpus.

Subfields: Applied Mathematics, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/48b2bde4af5ac22dc33def8576f188bd13711fe5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Feature Extraction by Non-Parametric Mutual Information Maximization (2003, methodology): “Based on the f-divergence family TORKKOLA (see Topsøe, 2000, and references therein), several further divergence measures can be written, one of which is the variational distance ).”
Symmetrizing the Kullback-Leibler Distance (2001, background): “What we call the Topsøe distance T (p0, p1) equals D(p0‖q) + D(p1‖q), where q = 12(p0 + p1).”
Inequalities for the L1 Deviation of the Empirical Distribution (2003, background): “It should be noticed that other refinements of Pinsker’s inequality considered in the literature (see [5], [6, Corollary 1.”
Fokker–Planck equations in the modeling of socio-economic phenomena (2017, background): “In particular, a class of relative entropies, known as symmetric Jensen–Shannon relative entropies,[73, 88, 93] seems useful to be studied in this context.”
Properties of Classical and Quantum Jensen-Shannon Divergence (2008, methodology): “[47] B. Fuglede and F. Topsøe.”
On Pinsker's and Vajda's Type Inequalities for Csiszár's $f$ -Divergences (2006, background): “A precise bound is available for the Capacitory divergence, for which Tops e [26] showed that C(P;Q) P1 n=1[n(2n 1)2 2n] 1V 2n = log 4 V 2 4 + V 2 log 2+V 2 V .”
Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning (2001, background): “The triangular discrimination (Topsøe, 2000) between two distributions p,q on the finite set S is given by d4(p,q) def = ∑ i∈S (pi −qi) pi +qi .”
Probability Product Kernels (2004, background): “Hellinger’s distance can be seen as a symmetric approximation to the Kullback-Leibler (KL) divergence, and in fact is a bound on KL, as shown in (Topsoe, 1999), where relationships between several common divergences are discussed.”
Nonparametric estimation of the likelihood ratio and divergence functionals (2007, background): “Different choices of ` result in many divergences that play important roles in information theory and statistics, including the variational distance, Hellinger distance, KL divergence and so on (see, e.g., [ 17 ]).”
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization (2008, background): “ny divergences that play important roles in information theory and statistics, including not only the KL divergence (1) but also the total variational distance, the Hellinger distance, and so on; see [24] for further examples. We begin by stating and proving a variational representation for the dive”
On Measures of Entropy and Information (2015, background): “JS0(A;B) = D(A ∥ B) JS 1 2 (A;B) = JS(A;B) JS1(A;B) = D(B ∥ A) 7Capacitory discrimination is defined as twice the Jensen-Shannon divergence by Topsøe [32] Jensen-Shannon entropy (generalized Jensen-Shannon divergence) [35, 34] JSΘ(A1;A2; · · · ;An) := ∑ α=1,n P(Θα) D ( Aα ∥ M ) , (26) P(Mx) = ∑ α P(”
On f-Divergences: Integral Representations, Local Behavior, and Inequalities (2018, neutral): “D], [60], [61], [62], [63]); 2) Inequalities which rely on the characterization of the exact locus of the joint range of f -divergences [34]; 3) f -divergence inequalities via functional domination ([58], [59, Section 3], [65], [66]); 4) Sharp f -divergence inequalities by using numerical tools for ”
Universal Estimation of Information Measures for Analog Sources (2009, background): “For more inequalities regarding divergence and related measures, see [34, 59, 85, 158, 238].”
Bhattacharyya and Expected Likelihood Kernels (2003, background): “Note that the Hellinger distance can be seen as a principled symmetric approximation of the Kullback Leibler (KL) divergence and in fact is a bound on KL as shown in [24] where relationships between many information theoretic divergences are characterized.”
Information, Divergence and Risk for Binary Experiments (2009, background): “Topsøe (2000) has presented some infinite series representations for capacitory discrimination in terms of triangular discrimination which lead to inequalities between those two divergences.”
A new metric for probability distributions (2003, background): “Topsøe, in [9], called it capacitory discriminationand introduced it from an information-transmission point of view.”
Jensen-Shannon divergence and Hilbert space embedding (2004, background): “the fact that √ JSD is a metric), [5] (some identities and inequalities), [6] (explicit proof of the metric property) and [7] (another independent explicit proof).”