Information theoretical estimators toolbox

By Zoltán Szabó. 2014

122 citations recorded by the source corpus.

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “However, the näıve implementation was fast enough for our purpose.”
Learning Theory for Distribution Regression (2014, methodology): “In our experiments we used the ITE toolbox (Information Theoretical Estimators; Szabó (2014)).14 6.1 Supervised Entropy Learning We compare the analysed MERR technique with Póczos et al. (2013)’s DFDR (kernel smoothing based distribution free distribution regression) method, on a benchmark problem”
A Principled Approach to Learning Stochastic Representations for Privacy in Deep Neural Inference (2020, methodology): “NEURAL NETWORK ARCHITECTURES Table 1 shows the DNNs used for the experiments.”
Shredder: Learning Noise to Protect Privacy with Partial DNN Inference on the Edge (2019, methodology): “Mutual Information (MI) is calculated using the Information Theoretical Estimators Toolbox’s [37] Shannon Mutual Information with KL Divergence.”
A Theory of Usable Information Under Computational Constraints (2020, methodology): “…information estimators in this low dimensional setting: KDE, the traditional kernel density estimator; the KSG estimator (Kraskov et al., 2004); the Mixed KSG estimator (Gao et al., 2017) and Partitioning, an adaptive partitioning estimator (Darbellay & Vajda, 1999) implemented by Szabó (2014).”
Bootstrap rank‐ordered conditional mutual information (broCMI): A nonlinear input variable selection method for water resources modeling (2016, methodology): “The broCMI algorithm was developed in Matlab using custom scripts (along with the requirement of the ITE Matlab toolbox [Szabo, 2014] for the estimation of CMI via the EA method).”
Estimating Mutual Information for Discrete-Continuous Mixtures (2017, methodology): “For all synthetic experiments, we compare our proposed estimator with a (fixed) partitioning estimator, an adaptive partitioning estimator [11] implemented by [47], the KSG estimator [27] and noisy KSG estimator (by adding Gaussian noise N(0, σ(2)I) on each sample to transform all mixed distribution”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “For more details, see the documentation of the ITE toolbox (Szabó, 2014).”