On the empirical estimation of integral probability metrics

By Bharath K. Sriperumbudur, Kenji Fukumizu, Arthur Gretton, B. Schoelkopf, Gert R. G. Lanckriet. 2012

346 citations recorded by the source corpus.

Given two probability measures, P and Q defined on a measurable space, S, the integral probability metric (IPM) is defined as γF(P,Q) = sup {∣∣∣∣ ∫

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “Thus, F should be restrictive enough for the empirical estimate to converge quickly (see, e.g., Sriperumbudur et al. (2012)).”
Minimax Estimation of Kernel Mean Embeddings (2016, background): “Remark 4 (KME expands small distances) For a Gaussian kernel, it is possible to show (Sriperumbudur et al., 2012, Example 3; Ramdas et al., 2015, Proposition 1) that ‖θ0 − θ1‖2Hk = C1 ( 1 − exp(−C2‖µ0 − µ1‖22) ) , where C1 and C2 are positive constants that depend only on σ2 and η2.”
On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions (2014, methodology): “3 in (Sriperumbudur et al. 2012a) also makes the latter choice, though only for verifying estimation error decay rate.”
Kernel Mean Shrinkage Estimators (2014, background): “Thus, the algorithms become less susceptible to the curse of dimensionality (see, e.g. , Wasserman (2006, Section 6.5) and Sriperumbudur et al. (2012)).”
On the optimal estimation of probability measures in weak and strong topologies (2013, background): “[15] proved a similar result for F BL with d = 1 which can be generalized to any d ≥ 2 using [31, Corollary 3.5].”
Learning Generative Models with Sinkhorn Divergences (2017, neutral): “g [34]), namely by considering a dual norm L(μ, ν) = ||μ − ν||B where ||ξ||B = sup {∫ X h(x)dξ(x) ; h ∈ B } .”
Computational Optimal Transport (2018, background): “13), and contrary to Wasserstein distances, the sample complexity does not depend on the ambient dimension E(|‖α̂n − β̂n‖k − ‖α− β‖k |) = O(n − (1)2 ); see [Sriperumbudur et al., 2012].”
McGan: Mean and Covariance Feature Matching GAN (2017, methodology): “By choosing F appropriately (Sriperumbudur et al., 2012; 2009), various distances between probability measures can be defined.”
Fisher GAN (2017, methodology): “Generalization bounds for the sample quality of the estimated Fisher IPM from samples from P and Q can be done akin to [11], with the main difficulty that for Fisher IPM we have to bound the excess risk of a cost function with data dependent constraints on the function class.”
Generalization Bounds for Domain Adaptation (2012, methodology): “Recently, Sriperumbudur et al. (2009, 2012) gave the further investigation and proposed an empirical method to compute the integral probability metric.”
On Regularization and Robustness of Deep Neural Networks (2018, background): “, 2017), the MMD interpretation yields a parametric statistical rate O(n−1/2) when learning from an empirical distribution with n samples, which is significantly better than the O(n−1/d) rate of the Wasserstein-1 distance for highdimensional data such as images (Sriperumbudur et al., 2012).”
Demystifying MMD GANs (2018, background): “The standard biased estimator of the MMD (Gretton et al., 2012, Equation 5), the IPM estimators of Sriperumbudur et al. (2012), and the empirical Wasserstein estimator studied by Bellemare et al. (2017) are all of this form.”
Sample Complexity of Sinkhorn Divergences (2018, background): “At the other end of the spectrum, the maximum mean discrepancy (MMD) (Gretton et al., 2006) is an integral probability metric (Sriperumbudur et al., 2012) on a reproducing kernel Hilbert space (RKHS) of test functions.”
Geometrical Insights for Implicit Generative Modeling (2017, background): “Comparable results for the 1-Wasserstein distance describe a convergence speed that quickly becomes considerably slower with the dimension d > 2 of the sample space X [50, 16, 18].”
Parametric Adversarial Divergences are Good Task Losses for Generative Modeling (2017, methodology): “• Wasserstein-1 distance induced by an arbitrary norm ‖·‖ and its corresponding dual norm ‖ · ‖∗ (Sriperumbudur et al., 2012): W (p||qθ) =̂ sup f :X→R ∀x∈X , ||f ′(x)||∗≤1 Ex∼p[f(x)]−Ex′∼qθ [f(x′)] (5) which can be interpreted as the cost to transport all probability mass of p into q, where ‖x− x′‖…”
Estimating individual treatment effect: generalization bounds and algorithms (2016, methodology): “Both the Wasserstein and MMD metrics have consistent estimators which can be efficiently computed for finite samples (Sriperumbudur et al., 2012), and have been used for various machine learning tasks in recent years (Gretton et al.”