The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal

By Jiantao Jiao, Weihao Gao, Yanjun Han. 2017

48 citations recorded by the source corpus.

We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over H\"older balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a new minimax lower bound over the H\"older ball, we show that the KL estimator is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter $s$ of the H\"older ball for $s\in (0,2]$ and arbitrary dimension $d$, rendering it the first estimator that provably satisfies this property.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Ewout van den Berg

In Statistics and Probability

Citations in the corpus

Estimating Information Flow in Neural Networks (2018, methodology): “, based on k nearest neighbours (kNN) [12] or kernel density estimation (KDE) techniques [13]) are applicable here by accessing the ‘noisy’”