On Measures of Entropy and Information

By A. Rényi. 1961

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Theory

Citations in the corpus

Fair Resource Allocation in Federated Learning (2019, background): “For this reason several popular fairness measurements have been proposed to balance between fairness and total throughput, including Jain’s index [19], entropy [33], max-min/min-max fairness [31], and proportional fairness [20].”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, neutral): “ν, the Renyi divergence of order λ [38] writes as dR,λ(μ1, μ2) := 1 λ−1 log ∫ Y g2(y) ( g1(y) g2(y) )λ dν(y).”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “• Rényi’s α-divergence (Rényi, 1961): Rα[p||q] = 1 α− 1 log ∫ p(ω)αq(ω)1−αdω.”
Rényi Divergence and Kullback-Leibler Divergence (2012, methodology): “Let θ ∈ [0, 1] denote the success probability of a binomial distribution Pθ = Bin(2, θ) on X = {0, 1, 2}.”
Information content and complexity of simulated soil water fluxes (2006, methodology): “Rényi (1961) introduced a generalization of the Shannon entropy — the Rényi entropy of the order α that allows weighing rare or frequent words differently.”
Effects of livestock breed and grazing intensity on grazing systems: 3. Effects on diversity of vegetation (2007, methodology): “The frequency distributions of patches were also used to calculate the heterogeneity of the sward structure in terms of divergence from an expected distribution, applying the order-one information index or mutual information (Renyi, 1961) on each frequency table, I 1⁄4 P F(i) ln [F(i)/P(i)], where F”
A review of multimodal image matching: Methods and applications (2021, methodology): “The use of Renyi entropy [351,352] has also attracted great attention in the registration problem, which is conducted with minimum spanning tree or spanning graphs [353], or by integrating with KLD [354] for better generalization.”
Statistical properties and different methods of estimation of Gompertz distribution with application (2018, background): “Some popular entropy measures are Rényi entropy (Rényi, 1961) or Shannon entropy (Shannon, 1951).”
On Measures of Entropy and Information (2015, background): “We’ll call these the Jeffreys and Jensen symmetrizations respectively [2], since a Jeffreys symmetrization of the relative entropy (13) leads to the Jeffreys entropy (16), and a Jensen symmetrization gives the Jensen-Shannon entropy (17).”