Entropy estimates of small data sets

By J. A. Bonachela, H. Hinrichsen, Miguel Angel Muñoz. 2008

118 citations recorded by the source corpus.

Estimating entropies from limited data series is known to be a non-trivial task. Naïve estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new ‘balanced estimator’ for entropy functionals (Shannon, Rényi and Tsallis) specially devised to provide a compromise between low bias and small statistical errors, for short data series. This new estimator outperforms other currently available ones when the data sets are small and the probabilities of the possible outputs of the random variable are not close to zero. Otherwise, other well-known estimators remain a better choice. The potential range of applicability of this estimator is quite broad specially for biological and digital data series.

Subfields: Molecular Biology, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/66878cf25563570934d98f86c336431e21ecff27

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

1. Martin A. Vinck
2. Bruno Hérault
3. Eric Marcon

In Statistical and Nonlinear Physics

1. Joseph Troy Lizier
2. Martin A. Vinck
3. Bruno Hérault
4. Eric Marcon

In Statistics and Probability

1. Joseph Troy Lizier
2. Bruno Hérault
3. Eric Marcon

Citations in the corpus

• Gamma-band synchronization in the neocortex: novel analysis methods and their application to sensory and motivational systems (2013, background): “Searching for an entropy estimator with minimum bias or distortion leads to large variance and asymptotical efficiency issues, and there is a general trade-off between the variance and bias of entropy estimators (Paninski, 2003; Bonachela et al., 2008; Schürmann, 2004).”
• JIDT: An Information-Theoretic Toolkit for Studying the Dynamics of Complex Systems (2014, background): “, Paninski (2003), Bonachela et al. (2008), though not yet implemented in JIDT.”
• entropart : An R Package to Measure and Partition Diversity (2015, methodology): “Correction of estimationbias is used to improve the estimation of entropy despite unobserved species and also mathematical issues (Bonachela, Hinrichsen, and Muñoz 2008).”