Lectures Notes in Information Theory

By Po-Ning Chen, Fady Alajaji. 2000

308 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/159fa026fcdee60ac08e1623f257c4106a1598ed

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation (2016, unknown): “where d2(p‖q) , p log pq + (1− p) log 1−p 1−q is the binary relative entropy [35].”
On the Robustness of Information-Theoretic Privacy Measures and Mechanisms (2018, background): “Observe that Lemma 3 is an immediate consequence of the data processing inequality (DPI) for f - information [60, Remark 2.3] and the fact that X 0 is a deterministic function of X .”
Statistical Windows in Testing for the Initial Distribution of a Reversible Markov Chain (2018, background): “The connections between notions of “distances” between distributions and sample complexity have been extensively studied, see e.g. (Polyanskiy and Wu, 2017) and references therein, and (Berend et al., 2014).”
Obfuscation via Information Density Estimation (2019, background): “By the concentration of the information density [63], we also know the probability that the information density is clipped is upper bounded, i.”
Understanding Variational Inference in Function-Space (2020, background): “Proposition 1 (Data processing inequality Polyanskiy and Wu, 2014, Thm 6.2) Let g be a measurable function, then D KL ( g ∗ P, g ∗ Q ) ≤ D KL ( P, Q ) , Background on Functional Variational Inference We consider the application of variational inference to regression.”
Information-theoretic analysis of generalization capability of learning algorithms (2017, background): “…mutual information between the overall input and output is upper bounded by the sum of the input-output mutual information of individual channels [20]; the formula for the capacity of input power constrained Gaussian channel; the fact that L S ( w j ) ’s are σ/ √ n subgaussian hence Var[ L S ( w…”