Natural Gradient Works Eciently in Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation (2019, methodology): “…second-order information through the Hessian matrix in Newton methods (Liu and Nocedal, 1989; Nocedal and Wright, 2006; Martens, 2010) or through the Fisher information matrix in natural gradient meth-ods (Amari, 1998; Roux et al., 2008; Pascanu and Ben-gio, 2014; Martens and Grosse, 2015).”
Trust-PCL: An Off-Policy Trust Region Method for Continuous Control (2017, methodology): “It is often more eﬀective to deﬁne an appropriate Riemannian metric that respects the loss surface (Amari, 2012), which allows much steeper descent directions to be identiﬁed within a local neighborhood ( e.g., Amari (1998); Martens & Grosse (2015)).”
Variational Inference: A Review for Statisticians (2016, positive): “In gradient-based optimization, the natural gradient accounts for the geometric structure of probability parameters (Amari 1982, 1998).”
Supervised Dimensionality Reduction via Distance Correlation Maximization (2016, methodology): “In Suzuki and Sugiyama [2013] (LSDR), the authors optimize this objective using a natural gradient based iterative solution on the Steifel manifold S md ( R ) via a line search along the geodesic in the direction of the natural gradient [Amari, 1998, Nishimori and Akaho, 2005].”
Natural continual learning: success is a journey, not (just) a destination (2021, background): “…distributions It can be shown that the direction of steepest decent for small step sizes is in this case given by (Kunstner et al., 2019; Amari, 1998) where F is the Fisher information matrix We thus get an update rule of the form which has units of [ θ ] and corresponds to a step in…”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, methodology): “An interesting alternative to variants of Newton’s method are variants of the natural gradient method (Amari, 1998), but like the basic Newton method it is computationally too expensive, requiring operations on a too large square matrix (number of parameters by number of parameters).”
Parallel training of Deep Neural Networks with Natural Gradient and Parameter Averaging (2014, background): “There are reasons from statistical learning theory, re-lated to the Natural Gradient idea [9], why we may want to set the learning-rate matrix E t to the inverse of the Fisher information matrix.”
Krylov Subspace Descent for Deep Learning (2011, methodology): “The use of the Fisher matrix in this way is known as Natural Gradient Descent (Amari, 1998); in Le Roux et al. (2007), a low-rank approximation of the Fisher matrix was used instead.”
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning (2019, background): “Furthermore, for any trajectory that starts at a point ( 1 [ 0 ] , 2 [ 0 ] ) within the WTA region to the left of this null cline, if J 1 [ 0 ] ≤ 1 [ 1 ] , then it will exit the WTA region at ( 1 [ 1 ] , 2 [ 1 ] ) or above, and thus hit a plateau that is at least as ﬂat.”
Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation (2010, methodology): “However, on a manifold, the natural gradient (Amari, 1998) gives the steepest direction.”
A Unifying Information-Theoretic Framework for Independent Component Analysis (2000, methodology): “However, there are a number of local learning rules for ICA such as that for the recurrent Herault-Jutten architecture and Linsker's (1997) network for Bell and Sejnowski's ICA learning rule.”
Applying the Episodic Natural Actor-Critic Architecture to Motor Primitive Learning (2007, background): “Natural gradients & ∇ θ J ( θ ) avoid this pitfall as demonstrated for supervised learning problems [3], and suggested for reinforcement learning in [2].”
Newton-Stein Method: An Optimization Method for GLMs via Stein's Lemma (2015, unknown): “Classical Newton’s Method (NM) and Natural Gradient (NG) descent can be recovered by simply taking Q to be the inverse Hessian and the inverse Fisher’s information at the current iterate, respectively [Ama98, Nes04].”
Multi-Agent Generative Adversarial Imitation Learning (2018, methodology): “…(MACK), is based on Actor-Critic with Kronecker-factored Trust Region (ACKTR, (Wu et al., 2017)), a state-of-the-art natural policy gradient (Amari, 1998; Kakade, 2002) method in deep RL. MACK uses the framework of centralized training with decentralized execution (Foerster et al., 2016);…”
Neural-network quantum state tomography (2017, methodology): “For the learning of the phases however, we instead implemented the natural gradient descent 35 , which revealed to be more eﬀective, though at the cost of increased computational resources.”
Efficient natural evolution strategies (2009, background): “…since the natural gradient is invariant w.r.t. the particular parameterization of the mutation distribution, it can cope with ill-shaped ﬁt-ness landscapes and provides isotropic convergence properties, which prevents premature convergence on plateaus and avoids overaggressive steps on ridges [1].”
Full Paper Reinforcement Learning for Imitating Constrained Reaching Movements (methodology): “In the actor part, the policy is improved by using the “natural” gradient descent [13] which is a steepest gradient descent algo-rithm with respect to the Fisher information matrix.”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, background): “Amari (1998) discussed the use of natural gradient in stochastic gradient optimization and pointed out that the steepest direction is given by the natural gradient when the parameter space is not Euclidean but has a Riemannian metric structure.”
Local minima and plateaus in hierarchical structures of multilayer perceptrons (2000, methodology): “It is known that a typical learning curve shows a plateau in the middle of training, which causes very slow decrease of the training error for a long time before a sudden exit from it (Amari, 1998; Saad & Solla, 1995).”