Information Maximization in Noisy Channels : A Variational Approach

40 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

NVAE: A Deep Hierarchical Variational Autoencoder (2020, background): “First, VAEs maximize the mutual information between the input and latent variables [29, 30], requiring the networks to retain the information content of the input data as much as possible.”
Unsupervised Control Through Non-Parametric Discriminative Rewards (2018, background): “…we lower bound the expectation term by replacing p ( s g | s T ) with a variational distribution q φ ( s g | s T ) with parameters φ following Barber & Agakov (2004), leading to Finally, we discard the entropy term H ( s g ) from (2) because it does not depend on either the policy parameters…”
An Information-Theoretic Analysis of Deep Latent-Variable Models (2017, background): “Barber & Agakov (2003) was the ﬁrst to introduce tractable variational bounds on mutual information, and made close analogies and comparisons to maximum likelihood learning and variational autoencoders.”
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning (2019, unknown): “However, for any distribution q, the following lower bound for −H(G | S) holds: −H(G | S) = E(G,S)∼pφt ,π [log q(G | S)] +DKL(p | q) ≥ E(G,S)∼pφt ,π [log q(G | S)] , where DKL denotes Kullback–Leibler divergence as discussed by Barber & Agakov [3].”