Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization

By XuanLong Nguyen, Martin J. Wainwright, Michael I. Jordan. 2008

920 citations recorded by the source corpus.

We develop and analyze M-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a nonasymptotic variational characterization of f -divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve optimal minimax rates for the likelihood ratio and the divergence functionals in certain regimes. We derive an efficient optimization algorithm for computing our estimates, and illustrate their convergence behavior and practical viability by simulations.

Subfields: Control and Systems Engineering, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

In Optimization & Training

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “The FactorVAE (Kim and Mnih, 2018) and the β-TCVAE (Chen et al., 2018) penalize the total correlation (Watanabe, 1960) with adversarial training (Nguyen et al., 2010; Sugiyama et al., 2012) or with a tractable but biased Monte-Carlo estimator respectively.”
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017, methodology): “While many model classes such as Pixel-RNNs (van den Oord et al., 2016b), PixelCNNs (van den Oord et al., 2016a), real NVP (Dinh et al., 2016) and Plug & Play generative networks (Nguyen et al., 2016) have been introduced and studied, the two most prominent ones are Variational Autoencoders (VAEs) (”
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (2016, methodology): “[25] derive a general variational method to estimate f -divergences given only samples from P and Q.”
Least Squares Generative Adversarial Networks (2016, background): “[23] pointed out that the objective of the original GAN [6] which is related to Jensen-Shannon divergence is a special case of divergence estimation, and generalized it to arbitrary f-divergences [22].”
Disentangling by Factorising (2018, methodology): “Having access to samples from both distributions allows us to minimise their KL divergence using the density-ratio trick (Nguyen et al., 2010; Sugiyama et al., 2012) which involves training a classifier/discriminator to approximate the density ratio that arises in the KL term.”
Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions (2011, methodology): “Recently, Nguyen et al. (2009, 2010) developed methods for estimating f -divergences using their variational characterization properties.”
On the empirical estimation of integral probability metrics (2012, background): “The empirical estimation of φ-divergences, especially the KL-divergence, has recently been studied in depth [30, 32, 52].”
Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning (2020, neutral): “2 Black-box estimator with MMD There are different choices for D(· ‖ ·) in equation 9, and multiple approaches to solve it [3, 31].”
AlgaeDICE: Policy Gradient from Arbitrary Experience (2019, positive): “To make this objective more approachable, we transform the f -divergence to its variational form (Nguyen et al., 2010) by use of a dual function x : S × A→ R: max π min x:S×A→R J̃D,f (π, x) := E(s,a)∼dπ [r(s, a)] + α · E(s,a)∼dD [f∗(x(s, a))]− α · E(s,a)∼dπ [x(s, a)] = E(s,a)∼dπ [r(s, a)− α · x(s,…”
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections (2019, methodology): “…been proposed [44], including the moment matching approach [17], probabilistic classiﬁcation approach [4, 7, 40], and ratio matching approach [22, 33, 45] The proposed DualDICE algorithm, as a direct approach for density ratio estimation, bears some similarities to ratio matching [33], which is…”
Continuous Doubly Constrained Batch Reinforcement Learning (2021, neutral): “More sophisticated measures V are discussed in the literature on empirical process theory for density and f-divergence estimation (Nguyen et al., 2010; Geer & van de Geer, 2000; van der Vaart et al., 1996).”
Hilbert Space Embeddings and Metrics on Probability Measures (2009, methodology): “Keywords: Probability metrics, Homogeneity tests, Independence tests, Kernel methods, Universal kernels, Characteristic kernels, Hilbertian metric, Weak topology.”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, background): “One possibility is use the variational formulation of the KL-divergence (Nguyen et al., 2010; Nowozin et al., 2016) that needs only samples from q φ ( z ) and p ( z ) to estimate a lower bound to KL( q φ ( z ) (cid:107) p ( z )).”
On Variational Bounds of Mutual Information (2019, methodology): “…in Belghazi et al. (2018) can be justiﬁed as unbiased estimates of the gradients of a different lower bound on MI. • We derive a new continuum of multi-sample lower bounds that can ﬂexibly trade off bias and variance, generalizing the bounds of (Nguyen et al., 2010; van den Oord et al., 2018).”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “To estimate it from samples, Kim and Mnih [3] rely on the density ratio trick [32, 33] which involves training a discriminator (see Section 2). β -TCVAE.”
Stabilizing Training of Generative Adversarial Networks through Regularization (2017, methodology): “We suggest to exploit properties of the maximizer ψ ∗ of F that can be characterized by Nguyen et al. (2010) The relevance of this becomes clear, if we apply the chain rule to ) as now we get a convenient cancellation of the Laplacians at The convexity of f c implies f c ≥ 0 , so that the weighting…”