Tempering Backpropagation Networks: Not All Weights are Created Equal
By N. Schraudolph, Terrence J. Sejnowski. 1995
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Grgoire Montavon
- J. Urgen Schmidhuber
- Wojciech Samek
- Yoshua Bengio
- K. Müller
- Kyunghyun Cho
- Behnam Neyshabur
- Ruslan Salakhutdinov
Citations in the corpus
- Deep learning in neural networks: An overview (2014, background): “The local gradient can be normalized based on the NN architecture (Schraudolph and Sejnowski, 1996), through a diagonalized”
- Measuring and regularizing networks in function space (2018, background): “…that many approaches to normalize and whiten activations or gradients have been forwarded in the literature (Raiko et al. (2012);Simard et al. (1998); Schraudolph & Sejnowski (1996); Crammer et al. (2009); Wang et al. (2013); LeCun et al. (1991); Schraudolph (1998); Salimans & Kingma (2016)).”
- Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent (2002, methodology): “A simple architecture-dependent technique such as tempering (Schraudolph & Sejnowski, 1996) should usually suffice to initialize p adequately; the fine tuning can be left to the SMD algorithm.”
- Local Gain Adaptation in Stochastic Gradient Descent (1999, methodology): “Normalization methods [2, 3, 4] calculate the optimal gain under simplifying assumptions — which may or may not model a given situation well.”
- Neural Networks: Tricks of the Trade (2012, methodology): “Physica D 22-D, 276–306 (1986) [15] Moody, J.”