Kullback-Leibler divergence estimation of continuous distributions

By Fernando Perez-Cruz. 2008 IEEE International Symposium on Information Theory, 2008

416 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/310974d3c141589a7800d737e5859b76676dcb5d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

On Estimating $L_2^2$ Divergence (2014, methodology): “This line of work has primarily focused on Kullback-Leibler, Renyi-α, and Csiszar f -divergences [11, 13, 14].”
Generating Differentially Private Datasets Using GANs (2018, methodology): “D and Inception Score, we compute image features by the Inception V3 network (Szegedy et al. 2016) and use inverse distances between features as simfunction. We implement the KL divergence estimator (Perez-Cruz 2008) and use´ k-d trees (Bentley 1975) for fast nearest neighbour searches. For privacy ”
Entropy and Kullback-Leibler divergence estimation based on Szegö's theorem (2009, background): “This problem , which is much harder than that of estimating these quantities for discrete random variables, has been studied , for instance, in [2] [3] [4] for the entropy, and in [5] [6] for the Kullback-Leibler divergence.”
Practical and Consistent Estimation of f-Divergences (2019, background): “While the majority of this focuses on ↵-divergences and closely related Rényi-↵ divergences [35, 37, 22], many works address specifically the KL-divergence [34, 39] with fewer considering f -divergences in full generality [28, 20, 26, 27].”
Identifying Best Interventions through Online Importance Sampling (2017, methodology): “We also note that estimates of the f -divergence measure can be had directly from empirical data (without the knowledge of the full distributions) using techniques like that of [30].”
Using Stacking to Average Bayesian Predictive Distributions (with Discussion) (2017, background): “…use the recommended Kullback–Leibler divergence, i.e., d(p, q) = KL(q, p), then there is substantial work that has focused on estimating the Kullback–Leibler divergence between two unknown densities based on samples from these densities (Leonenko et al., 2008; Pérez-Cruz, 2008; Bu et al., 2018).”
Least-squares two-sample test (2011, methodology): “Various methods for estimating the divergence functional have been studied so far (Darbellay & Vajda, 1999; Wang et al., 2005; Silva & Narayanan, 2007; Pérez-Cruz, 2008).”
On Estimating L 22 Divergence (2014, methodology): “This line of work has primarily focused on KullbackLeibler, Renyi-↵, and Csiszar f -divergences [12, 16, 17].”
Nonparametric Estimation of Renyi Divergence and Friends (2014, background): “A number of other authors have also proposed consistent estimators via the empirical CDF or histograms [25, 15].”