Information theoretic inequalities

By Amir Dembo, Thomas M. Cover, Joy A. Thomas. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

The Composition Theorem for Differential Privacy (2013, background): “As an aside, we mention that there is a strong tradition of such derivations in the information theory literature: the Fisher information inequality (Blachman, 1965; Zamir, 1998), the entropy power inequality (Stam, 1959; Blachman, 1965; Verdú & Guo, 2006), an extremal inequality involving mutual in”
PacGAN: The Power of Two Samples in Generative Adversarial Networks (2017, background): “, [26], [27], [28], [29], [30], [31], [32], [33], [34]) where operational interpretations of mutual information and corresponding data processing inequalities have given rise to simple proofs of strong technical results.”
The Cramer-Rao inequality for star bodies (2002, background): “It has previously been observed (see [ 4], 5], [6]) that there exists some connection between the subject of information theory and the Brunn-Minkowski theory.”
On Uncertainty Principle for Quaternionic Linear Canonical Transform (2013, background): “The uncertainty principle plays an important role in signal processing [17, 18, 31, 32, 52, 58, 59, 61, 72, 74, 83], and physics [3, 16, 39, 40, 41, 53,”
Hermite functions and uncertainty principles for the Fourier and the windowed Fourier transforms (2001, background): “We refer to the recent survey articles by Folland and Sitaram [11] and Dembo, Cover and Thomas [9] as well as the book of Havin and Jöricke [17] for various uncertainty principles of different nature which may be found in the literature.”
Information Inequalities for Joint Distributions, With Interpretations and Applications (2008, methodology): “This approach seems to have been first cast in probabilistic language by Dembo, Cover and Thomas [11], who further showed that an inequality of Szasz can be derived (and generalized) using Han’s inequality.”
Universal Estimation of Information Measures for Analog Sources (2009, background): “In addition to their prominent role in information theory, they have found numerous applications, among others, in probability theory [13, 19], ergodic theory [218], statistics [64, 142], convex analysis and inequalities [69], physics [25, 27, 147, 150], chemistry [79], molecular biology [270], ecol”
Divergence for s-concave and log concave functions (2013, background): “Applications are given in the theory of convex bodies.”
Some results concerning maximum Renyi entropy distributions (2005, unknown): “The approach of [5] is similar to that used by Cover and Thomas [6, p.234] in the Gaussian case.”
Visual Recognition with Humans in the Loop (2010, methodology): “Computationally, our method also has similarities to algorithms based on information gain, entropy calculation, and decision trees [14–16].”
An Extremal Inequality Motivated by Multiterminal Information Theoretic Problems (2006, background): “[12] in their “perturbation” proof of the classical EPI.”
Gradient of mutual information in linear vector Gaussian channels (2006, background): “Matrix Generalization of De Bruijn’s Identity For a multivariate density function , De Bruijn’s identity [2], [17]–[19] relates the derivative of the differential entropy to the Fisher information matrix defined as”
On the entropy of sums (2008, background): “The following three simple properties of mutual information can be found in elementary texts on information theory such as Cover and Thomas [7]: • When Y = X +Z, where X and Z are independent, then I(X; Y ) = H(Y ) − H(Z).”
Reverse Brunn–Minkowski and reverse entropy power inequalities for convex measures (2011, methodology): “Moreover, both the Brunn-Minkowski and the entropy power inequalities can be given similar proofs as limiting cases of Young’s inequality for convolution with sharp constant [24].”
An Extremal Inequality Motivated by Multiterminal Information-Theoretic Problems (2007, background): “[10] X = p 1 X + p X G; 2 [0; 1]: (122) Then fX g is a family of distributions indexed by 2 [0; 1] and connecting X (when = 0) with X G (when = 1).”
Representation formula for the entropy and functional inequalities (2010, background): “This inequality plays a central role in information theory, see [11] for an overview on the topic.”
On classes of non-Gaussian asymptotic minimizers in entropic uncertainty principles (2006, background): “can be expressed equivalently via the entropy power N(Xn) = 1 2πe exp ( 2 n H(Xn) ) (as given in [3]) by ( N(Xn)N(X̃n) ) 1 2 ≥ 1 2 .”