Information geometry of Boltzmann machines

By Shun-ichi Amari, K. Kurata, H. Nagaoka. 1992

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Revisiting Natural Gradient for Deep Networks (2013, background): “Natural gradient can be traced back to Amari’s work on information geometry (Amari, 1985) and its application to various neural networks (Amari et al., 1992; Amari, 1997), though a more in depth introduction can be found in Amari (1998); Park et al.”
Information geometry on hierarchy of probability distributions (2001, background): “…only for mathematical foundations of statistical inferences ([3], [12], [28] and many others) but also applied to information theory [5], [11], [25], [18], neural networks [6], [7], [9], [13], systems theory [4], [32], mathematical programming [33], statistical physics [10], [16], [38], and others.”
Natural Gradient Works Efficiently in Learning (1998, background): “It was shown in Amari et al. (1992) that the dynamic behavior of natural gradient in the Boltzmann machine is excellent.”
Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines (2013, background): “Namely, how to handle latent variables in pθ(x), a topic first discussed in [3].”
Training restricted Boltzmann machines: An introduction (2014, methodology): “For binary RBMs the entries of the Fisher information matrix (Amari et al., 1992; Desjardins et al., 2013; Ollivier et al., 2013) are given by”
Neural Learning in Structured Parameter Spaces - Natural Riemannian Gradient (1996, methodology): “The Jeffrey prior is given by 1 vIG(w)1 = Vn VC2(W){C1(W)}n-1.”
Training Restricted Boltzmann Machines (2015, methodology): “For binary RBMs the entries of the Fisher information matrix (Amari et al., 1992; Desjardins et al., 2013; Ollivier et al., 2013) are given by”