A General Class of Coefficients of Divergence of One Distribution from Another

By S. M. Ali, S. Silvey. 1966

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017, methodology): “Nowozin et al. (Nowozin et al., 2016) proposed to generalize Generative Adversarial Networks (Goodfellow et al., 2014) to f-divergences (Ali & Silvey, 1966) based on results by Nguyen et al. (Nguyen et al., 2010).”
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (2016, background): “A large class of different divergences are the so called f -divergences [5, 21], also known as the Ali-Silvey distances [1].”
On the empirical estimation of integral probability metrics (2012, background): “One popular family of divergence measures are the Ali-Silvey distances [1], also called the Csiszár’s φ-divergences [11], defined as”
Reinforcement Learning via Fenchel-Rockafellar Duality (2020, background): “The family of f -divergences, also known as Csiszár-Morimoto or Ali-Silvey divergences (Ali and Silvey, 1966), has been widely applied in many machine learning applications, including variational inference (Wainwright and Jordan, 2003), generative model estimation (Nowozin et al., 2016; Dai et al.,…”
Hilbert Space Embeddings and Metrics on Probability Measures (2009, methodology): “…divergence, the total variation distance, the Hellinger distance (Vajda, 1989) — these three are specific instances of the generalized φ-divergence (Ali and Silvey, 1966; Csiszár, 1967) — the Kolmogorov distance (Lehmann and Romano, 2005, Section 14.2), the Wasserstein distance (del Barrio…”
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization (2008, methodology): “For instance, in the asymptotic analysis of hypothesis testing, the Kullback-Leibler and Chernoff divergences control the decay rates of error probabilities (e.g., see Stein’s lemma [7] and its variants).”
Learning in Implicit Generative Models (2016, background): “A natural class of divergences to use are the f -divergences (or Ali-Silvey (Ali and Silvey, 1966) or Csiszar’s φ -divergence (Csiszàr, 1967)) since they are fundamentally linked to the problem of two-sample hypothesis testing (Liese and Vajda, 2008): f -divergences represent an integrated Bayes…”
Rényi Divergence and Kullback-Leibler Divergence (2012, background): “It has been argued that grouping observations together (by considering a coarser σ-algebra), should not increase our ability to distinguish between P and Q under any measure of divergence [29].”
On integral probability metrics, φ-divergences and binary classification (2009, background): “One of the widely studied and well understood families of distances/divergences between probability measures is the Ali-Silvey distance [4], also called the Csisz´ar’s φ -divergence [5], which is deﬁned as where M is a measurable space and φ : [0 , ∞ ) → ( , ∞ ] is a convex function.”
Computational Optimal Transport (2018, background): “Any entropy function φ induces a φ-divergence (also known as Csiszár divergence Ciszár [1967], Ali and Silvey [1966] or f -divergence) as follows.”
GOODNESS-OF-FIT TESTS VIA PHI-DIVERGENCES (2006, background): “Birkhäuser, Basel] for the Berk–Jones statistic applies to the whole family of statistics Sn(s) with s ∈ [−1,2].”
Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach (2016, methodology): “We begin by recalling divergence measures for probability distributions [1, 30].”
MINE: Mutual Information Neural Estimation (2018, methodology): “…MEASURES We close this section by pointing out that the previous construction can be extended to more general information measures based on so-called f -divergences (Ali & Silvey, 1966): Df (P || Q) := ∫ Ω f ( dP dQ ) dQ (17) indexed by a convex function f : [0,∞) → R such that f(1) = 0.”
On Pinsker's and Vajda's Type Inequalities for Csiszár's $f$ -Divergences (2006, methodology): “The class of -divergences was introduced by Csiszár [7], [17] and Ali and Silvey [18] and it has been used extensively both in information theory and in statistics—cf. [19]–[23].”
Semiparametric Counterfactual Density Estimation (2021, background): “In contrast to typical f -divergences [Ali and Silvey, 1966, Csiszár, 1967, Rényi et al., 1961, Sason and Verdú, 2016], we allow the function f to have two arguments, one for each distribution; this allows us to capture not only f -divergences but also other distances such as those based on Lp…”
A Game Theoretic Approach to Class-wise Selective Rationalization (2019, methodology): “As will be shown in the appendix A.2, equation (8) applies the f -divergence [1], which is a generalization to the KL-divergence as applied in equation (9).”