Information Theory: 1948-1998 - Guest Editorial

By Sergio Verdú. 2005

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

On the origin of long-range correlations in texts (2012, neutral): “In line with information theory, we treat a literary text as the output of a stationary and ergodic source that takes values in a ﬁnite alphabet and we look for information about the source through a statistical analysis of the text [23].”
Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “The rate distortion theory (Cover & Thomas, 2012) quantiﬁes the trade-off between the lossy compression rate R and the distortion D .”
Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model (2019, background): “For fixed θ , as K increases, KL(qθ |pθ ) decreases monotonically [5].”
Learning Generative ConvNets via Multi-grid Modeling and Sampling (2017, positive): “Let the distribution of Y τ be p τ , then KL ( p τ || p θ ) → 0 mono-tonically as τ → ∞ according to the second law of thermo-dynamics [3].”
Invariant Information Clustering for Unsupervised Image Classification and Segmentation (2018, background): “(1) is trivially solved by setting Φ to the identity function because of the data processing inequality [11], i.”
Shaking the foundations: delusions in sequence models for interaction and control (2021, background): “In fact, it is optimal in expectation with respect to the log-loss; and if the sequence is governed by latent variables (which is the case, for instance, when the sequence is exchangeable) then the scheme corresponds to Bayesian prediction [7].”