GAIT: A Geometric Approach to Information Theory

By Jose Gallego-Posada, Ankit Vani, Max Schwarzer, Simon Lacoste-Julien. 2019

12 citations recorded by the source corpus.

We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originally introduced in theoretical ecology to study the diversity of ecosystems. Based on this notion of entropy, we introduce geometry-aware counterparts for several concepts and theorems in information theory. Notably, our proposed divergence exhibits performance on par with state-of-the-art methods based on the Wasserstein distance, but enjoys a closed-form expression that can be computed efficiently. We demonstrate the versatility of our method via experiments on a broad range of domains: training generative models, computing image barycenters, approximating empirical measures and counting modes.

Subfields: Computational Theory and Mathematics, Computer Networks and Communications, Information Theory, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

1. Adji Bousso Dieng
2. Dan Friedman

In Computer Networks and Communications

1. Adji Bousso Dieng
2. Dan Friedman

In Information Theory

1. Adji Bousso Dieng
2. Dan Friedman

In Signal Processing

1. Adji Bousso Dieng
2. Dan Friedman

Citations in the corpus

• The Vendi Score: A Diversity Evaluation Metric for Machine Learning (2022, methodology): “Related to the metric we propose here is a similarity-sensitive diversity metric proposed in ecology by Leinster and Cobbold (2012), and which was introduced in the context of ml by Posada et al. (2020).”