On variational lower bounds of mutual information

By Ben Poole, Sherjil Ozair, Aäron van den Oord, Alexander A. Alemi, George Tucker. 2018

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, background): “In practice, as noted in [17], [18], maximizing I(Cψ(X);Eψ(X)) has several advantages: (a) Using different feature encodings allow us to capture different views and modalities of the data for flexibility of modelling [19], [20]; (b) The encoded data lies in a much lower dimensional latent space than”
Maximum Entropy Generators for Energy-Based Models (2019, methodology): “Training such a generator properly requires entropy maximization of the generator’s output distribution, for which we take advantage of recent advances in nonparametric mutual information maximization (Belghazi et al., 2018; Hjelm et al., 2018; Oord et al., 2018; Poole et al., 2018).”
Data-Efficient Mutual Information Neural Estimator (2019, methodology): “When limited number of samples are provided, estimations can suffer from high variance observed in [35].”
Wasserstein Dependency Measure for Representation Learning (2019, background): “A number of selfsupervised learning techniques can be shown to be maximizing a lower bound to mutual information between representations of different data modalities (Belghazi et al., 2018; van den Oord et al., 2018; Poole et al., 2018).”
Practical and Consistent Estimation of f-Divergences (2019, methodology): “For the particular case of the KL divergence, a similar approach has been heuristically applied independently by several authors for estimating the mutual information [36] and total correlation [6].”