Information geometry and its applications

By Felix Opitz. European Radar Conference, 2012

806 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Atomic and Molecular Physics, and Optics

View Paper: https://www.semanticscholar.org/paper/9f4f30780d9aa3ec21b5fd0c2d669ceb05178c07

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Atomic and Molecular Physics, and Optics

Citations in the corpus

The free energy principle made simpler but not too simple (2022, neutral): “In turn, this means that it is equipped with a metric and implicit information geometry [39, 40, 41].”
Sparse Variational Inference: Bayesian Coresets from Scratch (2019, background): “…posteriors deﬁnes a Riemannian statistical manifold M = { π w } w ∈ R N ≥ 0 with chart M → R N ≥ 0 , endowed with the Fisher information metric G [61, p. 33,34], For any differentiable curve γ : [0 , 1] → R N ≥ 0 , the metric deﬁnes a notion of path length, and a constant-speed curve of minimal…”
On the distance between two neural networks and the stability of learning (2020, positive): “Another classical technique is natural gradient descent (Amari, 2016), which replaces ‖∆W‖2F by ∆WTF∆W .”
Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models (2018, background): “This results in an update similar to the SGD update shown in (4), λt+1 = λt + αt [F(λt)] −1 ∇̂λL(λt), (8) where the stochastic gradient is scaled by the FIM.”
A short variational proof of equivalence between policy gradients and soft Q learning (2017, background): “(2006)); • finally, the theory of information geometry (Amari. (2016)) groups all three views and posits that there exists a general, dually flat Riemannian information manifold.”
A Brief Introduction to Machine Learning for Engineers (2017, background): “In particular, the following relationship holds [4] KL(p(x|η1)||p(x|η2)) = DA(η2, η1), (B.”
Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms (2018, background): “In other words, Fenchel-Young losses can be viewed as a “mixed-form Bregman divergence” (Amari, 2016, Theorem 1.1) where the ar-gument p in (4) is replaced by its dual point θ .”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “This equation can be interpreted as a projection from the data manifold to the policy model manifold [Amari, 2016].”
Information Geometry of Orthogonal Initializations and Training (2018, methodology): “In this context, the Fisher information matrix is deﬁned as [2, 7, 22], which can be expanding using the chain rule: Each block of the Fisher information matrix with respect to parameters a, b ∈ Θ can further be expressed as where the ﬁnal layer Hessian H g is deﬁned as ∇ 2 h g log p θ ( y x 0 ) .…”
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “These problems and the related challenges are often interconnected; thus, they may, in principle, belong to multiple categories.”
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (2018, background): “This is stated below where FIM is denoted by F(η), F(η)−1∇ηL(η) = ∇mL∗(m), (26) This relationship has been discussed in the earlier work of Hensman et al. (2012) and can also be found in Amari (2016).”
Online Learning via the Differential Privacy Lens (2017, neutral): “[8] Shun-ichi Amari.”