Transmission of Information: A Statistical Theory of Communications.

By G. Barnard, R. Fano. 1961

Subfields: Information Theory, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Theory

In Signal Processing

Citations in the corpus

Learning Robust and Multilingual Speech Representations (2020, neutral): “Mutual information measures relationships between random variables (Fano and Hawkins, 1961).”
Word Association Norms, Mutual Information, and Lexicography (1989, background): “What is "mutual information?" According to Fano (1961) , if two points (words), x and y, have probabilities P(x) and P(y), then their mutual information, I(x,y), is defined to be”
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training (2020, background): “Speciﬁcally, the negative value of I ( c 1 ; c 2 ; x ) indicates that the variable x enhances the correlation between { c 1 , c 2 } (Fano, 1963).”
Measuring Model Biases in the Absence of Ground Truth (2021, background): “In 1990, Church and Hanks [9] introduced a novel approach to quantifying associations between words based on mutual information [13, 23] and inspired by psycholinguistic work on word norms [28] that catalogue words that people closely associate.”
Predicting many properties of a quantum system from very few measurements (2020, background): “By relating the prediction task to a communication task [25], we establish fundamental lower bounds highlighting that classical shadows are (asymptotically) optimal.”
Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition (2000, unknown): “The mutual information between two random variables X and Y is The pointwise mutual information (Fano, 1961) 1 is a measure of how often two pointwisemutual information events x and y occur, compared with what we would expect if they were independent: I ( x , y ) = log 2 P ( x , y ) P ( x ) P ( y )…”
Experimental investigation of the entanglement-assisted entropic uncertainty principle (2010, unknown): “According to the Fano’s inequality relation [19]”
Probabilistic Relations between Words: Evidence from Reduction in Lexical Production (2008, background): “In Gregory et al. (1999) we considered the mutual information (Fano, 1961) of the target word and the following word.”
An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis (2000, methodology): “From information theory [3] we can define the lth band self-information of x x x and y y y as follows: I l (x x x) = 0 log p l and I l (y y y) = 0 log q l ; respectively: (2) Using (2), the relative entropy of y y y with respect to x x x can be defined by D(x x xky y y) = L l=1 p l D l (x x xky y…”
Predictability of population displacement after the 2010 Haiti earthquake (2012, background): “We get the maximum predictability, Πi, by solving a limiting case of Fano’s inequality (a relation derived from calculation of the decrease in information in a noisy information channel): Si = H(Πi) + (1 – Πi)log2(N – 1) (3) where H(Πi) = – Πilog2(Πi) – (1– Πi)log2(1 – Πi) (4) and N is the number of”
Synthetic review spamming and defense (2013, methodology): “The measure differs from the classic Pointwise Mutual Information (PMI) [5] in that Pi,j is not the joint probability of observing word wi and wj in a consecutive sentence pair, where wi and wj might co-occur in the same sentence.”
Pose Induction for Novel Object Categories (2015, methodology): “[20], who use Pointwise Mutual Information [12] (PMI) to counter similar biases, we normalize by the likelihood of randomly finding similar pose estimates for neighbors to yield -”
Information and Entropy Econometrics - A Review and Synthesis (2008, methodology): “Then, using Fano’s inequality (Fano, 1961) representing the relationship between the entropy (or the conditional entropy) and the probability of error is shown to be H(Pe) + Pe log(K − 1) ≥ H(X|Y ), where K is the dimension of X or Y .”