Methods of information geometry

By Shun-ichi Amari, H. Nagaoka. 2000

2,797 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Characteristic Kernels and Infinitely Divisible Distributions (2014, background): “vergence can be deﬁned in the set M(Rd) of kernel means: KL(mP ||mQ) = Z mP (x)log mP(x) mQ(x) dx, mP ,mQ ∈ M(R d). (15) This implies that the set M(Rd) can be an object for the information geometry (Amari and Nagaoka, 2000). The kernel mean map P → mP maps any m-ﬂat manifold to an m-ﬂat manifold in”
Boosting the Actor with Dual Critic (2017, methodology): “Since the dual variables are probabilistic distributions, it is natural to use KL -divergence as the prox-mapping to characterize the geometry in the family of parameters (Amari and Nagaoka, 1993; Nemirovski et al., 2009).”
New Insights and Perspectives on the Natural Gradient Method (2014, methodology): “Natural gradient descent is classically motivated as a way of implementing steepest descent2 in the space of realizable distributions3 instead of the space of parameters, where distance in the distribution space is measured with a special “Riemannian metric” (Amari and Nagaoka, 2007).”
Stabilizing Training of Generative Adversarial Networks through Regularization (2017, background): “Here, the principle of parameter estimation via divergence minimization is provably sound in that it achieves a well-defined limit [1, 21].”
A Distributional Approach to Controlled Text Generation (2020, background): “We follow Csiszár & Shields (2004) on this question, a problem that is at the core of the field of Information Geometry (Nielsen, 2018; Amari & Nagaoka, 2000).”
Biologically inspired protection of deep networks from adversarial attacks (2017, methodology): “To do so, we rely on the framework of information geometry and Fisher information (Amari & Nagaoka, 1993).”
Variable Metric Stochastic Approximation Theory (2009, methodology): “One example of considering variable metrics is the study of information geometry (Amari and Nagaoka, 1993), where the Fisher information matrix is used to define a metric tensor on a family of probability distributions.”
Online Optimization in Dynamic Environments (2013, background): “ described in this section is ecient and admits strong regrets bounds, it is applicable only for loss functions derived from exponential families. The basics of exponential families are described in [38,39], and Mirror Descent in this setting is explored in [40,41]. We assume some ˚: X !Rd which is ”
Information geometry on hierarchy of probability distributions (2001, background): “The present paper studies such a hierarchical structure and the related invariant " quasi-orthogonal " quantitative decomposition by using information geometry [3], [8], [12], [14], [28], [30].”
The geometry of proper scoring rules (2007, background): “These constructions and properties, which are special cases of the general theory of Lauritzen (1987a), make (P◦, g, ∇, ∇∗) a dually flat statistical manifold (Amari and Nagaoka, 1982; Lauritzen, 1987b; Amari and Nagaoka, 2000).”
Learning What’s Easy: Fully Differentiable Neural Easy-First Taggers (2017, background): “Let us start by writing the (standard) softmax in the following variational form (Amari and Nagaoka, 2001): where KL and H denote the Kullback-Leibler divergence and the entropy, respectively.”
Fast Variational Inference in the Conjugate Exponential Family (2012, methodology): “The natural gradient is given by [Amari and Nagaoka, 2007]: where G ( θ ) is the Fisher information matrix whose i , j th element is given by For exponential family distributions, this reduces to ∇ 2 θ ψ ( θ ) , where ψ is the log-normaliser.”
The geometry of relative arbitrage (2014, methodology): “To formulate it we need the Fisher information metric in information geometry [ANH07].”
Author manuscript, published in "International Journal of Computer Vision (2013)" International Journal of Computer Vision manuscript No. (will be inserted by the editor) Image Classification with the Fisher Vector: Theory and Practice (2013, positive): “From the theory of information geometry (Amari and Nagaoka, 2000), a parametric family of distributions U = { u λ , λ ∈ Λ } can be regarded as a Riemanninan manifold M Λ with a local metric given by the Fisher Information Matrix (FIM) F λ ∈ R M × M : Following this observation, Jaakkola and…”
Transferring Knowledge across Learning Processes (2018, background): “Riemann manifolds have also garnered attention in the context of optimization, as a preconditioning matrix can be understood as the instantiation of some Riemann metric (Amari & Nagaoka, 2007; Abbati et al., 2018; Luk & Grosse, 2018).”