Nearest Neighbor Estimates of Entropy

By Harshinder Singh, N. Misra, V. Hnizdo, A. Fedorowicz, E. Demchuk. 2003

266 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Organic Chemistry, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3adde98a328a48b5a49c23fa1ae6c3c89056b9f1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

Practical Bounds on Image Denoising: From Estimation to Information (2011, methodology): “This was later extended by considering -nearest neighbor distances in [ 36 ]–[38] (see also [39]).”
Benchmarking for Steganography (2008, background): “The only estimator that can provide accurate results in high dimensional spaces is the kNN estimator [4, 37], which we now briefly describe.”
A comparative review of dimension reduction methods in approximate Bayesian computation (2012, methodology): “ﬀuse posterior sample, whereas low entropy is obtained from a posterior which is more precise in nature. Nunes and Balding (2010) estimate entropy using the unbiased kth nearest neighbor estimator of Singh et al. (2003). For a weighted posterior sample, (w1,θ1),...,(wn,θn), where P i w i =1, this es”
Demystifying fixed k-nearest neighbor information estimators (2016, methodology): “Consistency of the KL estimator has been established for k = 1 by the original authors [27] and for general k by [49] and the rate of convergence of the bias and variance has been established (for a certain large class of smooth pdfs with unbounded support, including the Gaussian) only for one-dimen”
Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation (2016, background): “For k = 1 this was proved in the original paper of [15], which later was extended in [32, 9] to general k.”
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training (2021, methodology): “To handle this issue, we employ the simpliﬁed version of particle-based entropy estimator (Beirlant et al., 1997; Singh et al., 2003) (see the supplementary material for more details): where (cid:98) H denotes the particle-based entropy estimator and s ki is the k -th nearest neighbor ( k -NN) of s…”
Non-linear Causal Inference using Gaussianity Measures (2014, methodology): “GR-AN also performs better than GR-ENT, a method that uses, instead of statistical tests of Gaussianity, a non-parametric estimator of the entropy (Singh et al., 2003).”