Natural conjugate gradient training of multilayer perceptrons

21 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Revisiting Natural Gradient for Deep Networks (2013, background): “Such large steps may be present with SGD, possibly yielding the model to overﬁt (e.g. getting forced into some quadrant of parameter space based only on a few examples) resulting in different models at the end.”
Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes (2010, methodology): “This choice of geometry between Euclidean and Riemannian is, however, independent of the choice of the optimisation algorithm, and recently several authors have combined conjugate gradient methods with the Riemannian or natural gradient (Smith, 1993; González and Dorronsoro, 2008; Honkela et al., 20”