Fisher information inequalities and the central limit theorem

By Oliver Johnson, Andrew R. Barron. 2001

186 citations recorded by the source corpus.

Abstract.We give conditions for an O(1/n) rate of convergence of Fisher information and relative entropy in the Central Limit Theorem. We use the theory of projections in L2 spaces and Poincaré inequalities, to provide a better understanding of the decrease in Fisher information implied by results of Barron and Brown. We show that if the standardized Fisher information ever becomes finite then it converges to zero.

Subfields: Electrical and Electronic Engineering, Mathematical Physics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/3053cd612c9b1b0fa50852b674727b62f126496a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Mathematical Physics

In Statistical and Nonlinear Physics

Citations in the corpus

Fokker–Planck equations in the modeling of socio-economic phenomena (2017, background): “58) with the Hellinger distance.[68] In view of expression (3.”
On the power of Chatterjee’s rank correlation (2020, methodology): “(b) Using Lemma A.1 in Johnson and Barron (2004) and Lemma I.2.4.a in Hájek and Šidák (1967) yields limx→∞ fk(x) = 0 and E{ρk(X(k))} = 0.”
Stein’s density approach and information inequalities (2012, background): “this is a significant improvement on the constant in [20, 35].”
Rényi divergence and the central limit theorem (2016, background): “ts of such strong CLT’s, including the non-i.i.d. situation and the problem of rates or Berry-Esseen bounds, were studied by many authors, and we refer an interested reader to [Li], [S-M], [A-B-B-N], [B-J], [J], [B-C-G2-4], [B-C-K], [B-C]. As for convergence in Dα with α>1, not much is known so f”
Automated Scalable Bayesian Inference via Hilbert Coresets (2017, background): “This norm is a weighted version of the Fisher information distance (Johnson and Barron, 2004).”
Rate of convergence and Edgeworth-type expansion in the entropic central limit theorem (2011, background): “case as above, these authors have obtained an expected asymptotic bound D(Zn) = O(1/n) under the hypothesis that the distribution of X1 admits an analytic inequality of Poincaré-type (in [15], a restricted Poincaré inequality is used).”
On Maximum Likelihood Training of Score-Based Generative Models (2021, background): “More generally, various identities and inequalities involving the (relative) Shannon entropy and (relative) Fisher information have found use in proofs of the central limit theorem [Johnson and Barron, 2004].”
Log-Concavity and Strong Log-Concavity: a review (2014, background): “…beginnings in the work of Linnik (1959), Brown (1982), and Barron (1986), but has interesting cross-connections with log-concavity in the more recent papers of Johnson and Barron (2004), Carlen and Soffer (1991), Ball, Barthe and Naor (2003), Artstein et al. (2004a), and Artstein et al. (2004b).”
Central limit theorem and convergence to stable laws in Mallows distance (2004, methodology): “As in Johnson and Barron (2004), we examine this difference sequence, to show that its convergence implies convergence of D(Tk) to zero.”
Berry–Esseen bounds in the entropic central limit theorem (2011, unknown): “es, for example, when X 1 has a distribution satisfying an integro-diﬀerential inequality of Poincar´e-type. These results are due to Artstein, Ball, Barthe and Naor [A-B-B-N], and Barron and Johnson [B-J]; cf. also [J]. Recently, an exact asymptotic for D(Sn) has been studied in [B-C-G1]. If the en”
Maximum Likelihood Training of Score-Based Diffusion Models (2021, background): “More generally, various identities and inequalities involving the (relative) Shannon entropy and (relative) Fisher information have found use in proofs of the central limit theorem [24].”
Stein's method for comparison of univariate distributions (2014, background): “There is an important connection between such constants and Poincaré / variance bounds or spectral gaps, as already noted for example in [23, 56, 54, 61, 72].”
Entropy jumps for isotropic log-concave random vectors and spectral gap (2012, result): “In the paper [16], Johnson and Barron obtained a result similar to (1.”