Information Theory: Multi-terminal systems

By Imre Csiszár, J. Körner. 2011

1,903 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Theory

Citations in the corpus

Continuous Doubly Constrained Batch Reinforcement Learning (2021, positive): “…γ)2 Eπb(s) √ KL ( πb(a|s), πφ(a|s) ) ≥ J(πb)− r∗√ 2(1− γ)2 √ Eπb(s) [ KL ( πb(a|s), πφ(a|s) )] where the second inequality follows from a standard relation between the total-variation and KL divergences (c.f. Csiszár & Körner (2011)), and the last one is an application of Jensen’s inequality.”
Calibration, Entropy Rates, and Memory in Language Models (2019, background): “(Pinsker’s Inequality (Csiszar & Körner, 2011)) Suppose that for all w1:T , f(w1:T ) B.”
Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation (2014, background): “In view of the Pinsker inequality D ( P Q ) ≥ 2 TV 2 ( P, Q ) [CK82, p. 58] as well as the continuity property of entropy with respect to the total variation distance: we conclude that the best lower bound given by the two-point method, i.e., the supremum in (34), is on the order of log k √ n .”
On the limits of communication with low-precision analog-to-digital conversion at the receiver (2009, background): “While [28] provides this bound for a discrete channel, its extension to continuous alphabet channels has been established in [29].”
Constant Composition Distribution Matching (2015, methodology): “We measure the distance between the matcher output distribution and the desired distribution by normalized informational divergence [1, p. 7].”
Fundamental Limits of Communication With Low Probability of Detection (2015, methodology): “Proof: We have the following identity (see [20]): By the Kuhn-Tucker conditions for channel capacity [8], where equality holds if x ∈ supp ( P ∗ ) .”
Multinomial Adversarial Networks for Multi-Domain Text Classification (2018, background): “˜2 Neyman(k) is the Neyman ˜2 divergence (Nielsen and Nock,2014). The proof of both theorems can be found in the Appendix. Consequently, by the non-negativity and joint convexity of the f-divergence (Csiszar and Korner, 1982), we have: Corollary 1. The optimum of JD Fs is NlogN when using NLL loss, ”
K-polystability of Q-Fano varieties admitting Kähler-Einstein metrics (2012, background): “ of the ﬁrst inequality of the Theorem. The second inequality then follows immediately from the classical Csiszar-Kullback-Pinsker inequality between the relative entropy and the total variation norm [17]. As for the equality case it follows, just as in the second proof of Theorem 1.1, from the equa”
On Rényi measures and hypothesis testing (2011, background): “The first inequality follows from Theorem 1, and the second from the data processing inequality for the KullbackLeibler divergence [18].”
Information Theory of Decisions and Actions (2011, background): “For instance, it was noticed by Berger (2003) that biology might be using non-separable information in the sense of using joint source-channel coding (Csiszár and Körner 1986).”