How to Train Neural Networks

By Ralph Neuneier, Hans-Georg Zimmermann. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Robust Optimization for Deep Regression (2015, methodology): “Robust estimation methods, within our context, can be found in the literature for training artificial neural networks [32] or Hopfield-Tank networks [10], but not for deep networks.”
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods (2018, methodology): “These methods date back many years (e.g. vario-eta Neuneier & Zimmermann, 1998), and many variants have recently been developed, including AdaGrad (Duchi et al., 2011), RMSprop (Hinton et al., 2012) and Adam (Kingma & Ba, 2015).”
Batch reinforcement learning on the industrial benchmark: First experiences (2017, background): “Neither for RPROP, nor Vario-Eta [23], nor momentum-backprop stable training behaviors have been observed.”
Learning Long Term Dependencies with Recurrent Neural Networks (2006, unknown): “4 and 3). The networks were trained with BPTT in combination with pattern-by-pattern learning [ 12 ].”
Local Gain Adaptation in Stochastic Gradient Descent (1999, methodology): “Normalization methods [2, 3, 4] calculate the optimal gain under simplifying assumptions — which may or may not model a given situation well.”
Centering Neural Network Gradient Factors (1996, unknown): “Vario-· [12, 13, page 48].”
Interpretable Policies for Reinforcement Learning by Genetic Programming (2017, methodology): “For training, we used the Vario-Eta algorithm (Neuneier and Zimmermann, 2012).”
Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies (2016, methodology): “f 30;30gfor CPSU. All networks ~gconsist of two hidden layers with 20 hidden neurons each with arctan activation functions. The network training has been conducted by applying the Vario-Eta algorithm [26] and splitting the data sets into 90;000 training and 10;000 validation patterns. The training o”