Effect of Batch Learning in Multilayer Neural Networks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks (2013, background): “, N2, where r · r = δαβ , with all other connectivity modes a and b set to zero (see [20] for solutions to a partially overlapping but distinct set of initial conditions, further discussed in Supplementary Appendix A).”
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization (2018, methodology): “The dynamics of optimization was studied in Fukumizu (1998) and Saxe et al. (2013), for linear networks. Like ours, these works analyze gradient descent through its corresponding differential equations. Fukumizu (1998) focuses on linear regression with `2 loss, and does not consider the effect of va”
Minnorm training: an algorithm for training over-parameterized deep neural networks (2018, background): “However, the loss surface for this network is non-convex [3, 12, 24] and the solution manifold for this simple case is the hyperbola w 1 w 2 = 1 .”
On the information bottleneck theory of deep learning (2018, methodology): “To search for these mechanisms, we turn to a tractable model system: deep linear neural networks (Baldi and Hornik 1989, Fukumizu 1998, Saxe et al 2014).”