Information Theory: 1948-1998 - Guest Editorial

By Sergio Verdú. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Theory

Citations in the corpus

Measuring Data (2022, background): “Perplexity (language ): Historically, perplexity was used in information theory to reﬂect how well a probability distribution predicts a sample (Thomas and Joy, 2006).”
Generative Image Modeling Using Spatial LSTMs (2015, methodology): “ pixels (that is, gradients were approximated as in backpropagation through time [32]). The right column in Table 1 shows average log-likelihood rates for both models. Analogously to the entropy rate [3], we have for the expected log-likelihood rate: lim N!1 E logp(x)=N2 = E[logp(x ijjx <ij)], (8”
Generative Adversarial Imitation Learning (2016, neutral): “For all s and a , the log-sum inequality [6] = ¯ H ( ρ π ) , and second, = − s,a ρ π ρ ( s, a ) log π ρ ( a | s ) Proof of Proposition 3.2.”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, methodology): “When ρ admits a conditional probability distribution, We now postulate two unknowns: a pseudo-count function ˆ N n ( x ) , and a pseudo-count total ˆ n .”
Advances and Open Problems in Federated Learning (2019, background): “However usually in information theory, the compression guarantees are instance specific and depend on the entropy of the underlying distribution [122].”
Wireless Network Intelligence at the Edge (2018, methodology): “ould not exceed a given wireless channel capacity C bits/sample, i.e., R(D) C. To elaborate, when the MSI is treated as an IID. Gaussian source with variance ˙2, the Shannon distortion-rate function [91] is given as R(D) = 1=2 log2(˙2=D): (20) Using a uniform scalar quantizer with sufﬁciently small ”
On the Capacity of Secure Distributed Matrix Multiplication (2018, methodology): “…get (cid:101) Since there are possible subsets L of servers of size (cid:96) , we sum up their entropy and have, Rearranging (19), we have, where in ( a ) we apply Han’s inequality [13, Chapter 17 ] to bound the second term and ( b ) follows by bounding the joint entropy using the sum of entropies.”
Algorithms and Theory for Multiple-Source Adaptation (2018, background): “The ﬁrst term can be written in terms of the unnormalized relative entropy: x,y The unnormalized relative entropy B ( · (cid:107) · ) is jointly convex [Cover and Thomas, 2006], 2 thus B ( K z (cid:107) J z ) is convex as the composition of the unnormalized relative entropy with afﬁne functions…”
Learning Adversarially Fair and Transferable Representations (2018, positive): “…express it as The statistical distance (a.k.a. total variation distance) be-tween distributions is deﬁned as the maximum attainable test discrepancy (Cover & Thomas, 2012): When learning fair representations we are interested in the distribution of Z conditioned on a speciﬁc value of group…”
Extremal Mechanisms for Local Differential Privacy (2014, methodology): “Notice that I (X;Y ) ≤ H (X) and I (X;Y ) is convex in Q (Cover and Thomas, 2012).”
Dependence Makes You Vulnberable: Differential Privacy Under Dependent Tuples (2016, background): “5 and H(·) denotes the entropy (information) of a random variable [10].”