Dual representation of Φ-divergences and applications

By Amor Keziou. 2003

78 citations recorded by the source corpus.

Subfields: Applied Mathematics, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/124d9691a077f6130c23899331e63a7d9cd1e1df

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

Mutual Information Neural Estimation (2018, methodology): “We will primarily work with the Donsker-Varadhan representation (Donsker & Varadhan, 1983), which results in a tighter estimator; but will also consider the dual f -divergence representation (Keziou, 2003; Nguyen et al., 2010; Nowozin et al., 2016).”
Divergences and Duality for Estimation and Test under Moment Condition Models (2010, methodology): “se of parametric models of densities, White (1982) studied the asymptotic properties of the parametric maximum likelihood estimate and the parametric likelihood ratio statistic under misspeciﬁcation; Keziou (2003) and Broniatowski and Keziou (2009) stated the consistency and obtained the limiting di”
Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching (2012, methodology): “In this section, we first review a general framework of approximating the f -divergences (Ali and Silvey, 1966; Csiszár, 1967) via Legendre-Fenchel convex duality (Keziou, 2003; Nguyen et al., 2010).”
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization (2008, methodology): “ R φ(dQ/dP)dP, where φis a convex function of the likelihood ratio dQ/dP. Our estimation method is motivated by a non-asymptotic characterization of f-divergence, due independently to several authors [5, 14, 19]. Roughly speaking, the main theorem in [19] states that that there is a correspondence b”
On empirical likelihood for semiparametric two-sample density ratio models (2008, methodology): “(2.11) On the other hand, using the so-called “dual representation of -divergences” (see Keziou, 2003, Theorem 2.1; Broniatowski and Keziou, 2006, Theorem 4.4) and choosing the class of functions F := {x → ′(m( , x)); ∈ }, we can prove the equality sup ∈ {∫ f ( , x) dQ(x) − ∫ g ( , x) dP(x) } = ∫ (…”
Convex Formulation for Learning from Positive and Unlabeled Data (2015, methodology): “To cope with this problem, in Keziou (2003) and Nguyen et al. (2007), the following objective function for density ratio estimation was introduced: sup g ∫ g(x)p(x|y = 1)dx− ∫ f∗(g(x))p(x)dx, (10) wheref(t) is a convex function such thatf(1) = 0, and f∗(z) = supt tz − f(t) denotes its Fenchel dual.”
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation (2012, background): “This expression can also be obtained based on the fact that the f -divergence D ( P k P ′ ) is lower-bounded via the Legendre-Fenchel convex duality (Rockafellar, 1970) as follows (Keziou, 2003; Nguyen et al., 2007): where f ∗ is the convex conjugate of convex function f deﬁned at (2).”
Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation (2012, unknown): “(32), we have the following lower bound (Keziou 2003): ASC f (p ∗ nu‖p∗ de) = − inf g ASC ′ f (g),”
Information, Divergence and Risk for Binary Experiments (2009, methodology): “One can obtain a variational representation for If by substituting a variational representation for f into the definition of If (Keziou, 2003a,b; Broniatowski, 2004; Broniatowski and Keziou, 2009).”
Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation (2010, background): “The convex duality (Boyd & Vandenberghe, 2004) gives the variational representation (Keziou, 2003; Nguyen, Wainwright, & Jordan, 2010) of SMI as where inf g is taken over all measurable functions on ( D X × D Y , B X × B Y ) and This can be checked as follows.”
Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization (2007, methodology): “The variational representation of divergences has been der ive independently and exploited by several authors [5, 11, 14].”
On integral probability metrics, φ-divergences and binary classification (2009, background): “Consider the variational form of Dφ [24], [26], [28] given by”