On Choosing and Bounding Probability Metrics

By Alison L. Gibbs, Francis Edward Su. 2002

1,510 citations recorded by the source corpus.

When studying convergence of measures, an important issue is the choice of probability metric. We provide a summary and some new results concerning bounds among some important probability metrics/distances that are used by statisticians and probabilists. Knowledge of other metrics can provide a means of deriving bounds for another one in an applied problem. Considering other metrics can also provide alternate insights. We also give examples that show that rates of convergence can strongly depend on the metric chosen. Careful consideration is necessary when choosing a metric.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

On the empirical estimation of integral probability metrics (2012, background): “See [18] for a detailed discussion of the relation between various metrics.”
Agile Autonomous Driving using End-to-End Deep Imitation Learning (2017, background): “Let DW (·, ·) denote the Wasserstein metric [7]: for two probability distributions p and q defined on a metric space M with metric d,”
Hilbert Space Embeddings and Metrics on Probability Measures (2009, methodology): “By Theorem 4 in Gibbs and Su (2002), TV and W are related as W (P,Q) ≤ diam(M)TV (P,Q), (55) which means W and TV are comparable if M is bounded.”
Rényi Divergence and Kullback-Leibler Divergence (2012, methodology): “[24]: D1/2(P‖Q) = −2 ln ( 1− Hel (2)(P,Q) 2 ) .”
On integral probability metrics, φ-divergences and binary classification (2009, background): “ KL. Therefore, using these bounds, one can obtain a consistent estimate of a lower bound on TV and KL. The bounds in (46) and (47) also translate to lower bounds on other distance measures on P. See [57] for a detailed discussion on the relation between various metrics. To summarize, in this sectio”
Computational Optimal Transport (2018, background): “2: Diagram of relationship between divergences (inspired by Gibbs and Su [2002]).”
Metrics for Finite Markov Decision Processes (2004, background): “Sections 2 and 3 provide the deﬁnitions and theoretical results required to construct our metrics.”
Inverse problems in spaces of measures (2013, background): “On the other hand, Radon measures are well-studied objects and many notions of convergence are available (see, for instance, [24, 27]) for which one can hope that it holds for a numerical algorithm.”
Stochastic Gradient Markov Chain Monte Carlo (2019, methodology): “The most common distance used is the Wasserstein distance (Gibbs and Su 2002), primarily because it is particularly amenable to analysis.”
A Comparative Analysis of Expected and Distributional Reinforcement Learning (2019, background): “The theoretical properties of some probability distribution metrics have been previously explored by (Gibbs and Su 2002), and the Wasserstein distance in particular studied further in the context of MDPs (Ferns et al. 2012) as well as in the generative model literature (Arjovsky, Chintala, and…”
Transportation-information inequalities for Markov processes (2007, neutral): “The known inequality (see Gibbs and Su [21]) is 1 4 ‖ν − μ‖(2)TV ≤ d(2)H(ν, μ)[2 − d(2)H(ν, μ)] and the above right-hand side is exactly 1 − [μ(√f)]2 = Varμ( √ f).”
Stein's method for steady-state diffusion approximations of $M/Ph/n+M$ systems (2015, methodology): “}. (1.3) In this case, (1.1) provides rates of convergence for stationary distributions under the Wasserstein metric [45]; convergence under Wasserstein metric implies the convergence in distribution [24]. In [14], an algorithm was developed to compute the stationary distribution of the diffusion pr”
Kullback-Leibler divergence constrained distributionally robust optimization (2012, methodology): “Klabjan et al. (2012) considered the inventory management problem where the ambiguity of the demand distribution for a single item is depicted via the histogram and the χ2-distance.”