Divergence estimation of continuous distributions based on data-dependent partitions

By Qing Wang, Sanjeev R. Kulkarni, Sergio Verdú. 2005

208 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

On the empirical estimation of integral probability metrics (2012, background): “The empirical estimation of φ-divergences, especially the KL-divergence, has recently been studied in depth [30, 32, 52].”
Hilbert Space Embeddings and Metrics on Probability Measures (2009, methodology): “Keywords: Probability metrics, Homogeneity tests, Independence tests, Kernel methods, Universal kernels, Characteristic kernels, Hilbertian metric, Weak topology.”
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization (2008, methodology): “More recent recent work by Wang et al. [29] has developed algorithms for estimating the KL divergence between a pair of continuous distributions P and Q, based on building data-dependent partitions of equivalent (empirical) Q-measure.”
On integral probability metrics, φ-divergences and binary classification (2009, methodology): “Overall, the results in this section show that IPMs (especia lly the Wasserstein distance, Dudley metric and MMD) are easier to estimate than the KL-divergence and the IPM estimators exhibit better convergence behavior [17], [35].”
Kullback-Leibler divergence estimation of continuous distributions (2008, methodology): “In these figure we can see that the proposed estimator is more accurate than the one in [21], as it converges faster to the true divergence as the number of samples increases.”
Nonparametric Estimation of Renyi Divergence and Friends (2014, background): “A number of other authors have also proposed consistent estimators via the empirical CDF or histograms (Wang et al., 2005; Pérez-Cruz, 2008).”
Demystifying fixed k-nearest neighbor information estimators (2016, background): “[56] Qing Wang, Sanjeev R Kulkarni, and Sergio Verdú.”
Least-squares two-sample test (2011, methodology): “Various methods for estimating the divergence functional have been studied so far (Darbellay & Vajda, 1999; Wang et al., 2005; Silva & Narayanan, 2007; Pérez-Cruz, 2008).”
Parametric Bayesian Estimation of Differential Entropy and Relative Entropy (2010, background): “explored a number of data-dependent partitioning approaches for relative entropy between any two absolutely continuous distributions [16].”
Minimax Rate-optimal Estimation of KL Divergence between Discrete Distributions (2016, background): “There has been several attempts to estimate the KL divergence for the continuous case, see [4]–[7] and references therein.”
Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances (2009, methodology): “According to Algorithm A [38], the divergence is estimated by”
Empirical estimation of entropy functionals with confidence (2010, methodology): “The authors of these works [40, 17, 13, 45] sestablish that the estimators they propose are asymptotically unbiased and consistent.”
Entropy and Kullback-Leibler divergence estimation based on Szegö's theorem (2009, background): “This prob lem, which is much harder than that of estimating these quantities for discrete random variables, has been stud ied, for instance, in [2,3,4] for the entropy, and in [5,6] for the Kullback-Leibler divergence.”