Local minima in training of deep networks

44 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Visualizing the Loss Landscape of Neural Nets (2017, background): “[37] show counterexamples that achieve “bad” local minima for toy problems.”
Recovery Guarantees for One-hidden-layer Neural Networks (2017, background): “(Swirszcz et al., 2016) present examples showing that for this to be true additional assumptions on the data, initialization schemes and/or the model classes have to be made.”
An empirical analysis of the optimization of deep network loss surfaces (2016, methodology): “0) and observe the loss surface behaviours (We used same initialization distribution as (Swirszcz et al., 2016).)”
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks (2017, background): “(1)), there are many spurious local minima (e.g., [Auer et al., 1996, Swirszcz et al., 2016]).”
Sharp Minima Can Generalize For Deep Nets (2017, background): “In the spirit of (Swirszcz et al., 2016), our work indicates that more care is needed to define flatness to avoid degeneracies of the geometry of the model under study.”
Global optimization issues in deep network regression: an overview (2018, background): “On the other hand recently in [134] several concrete examples of datasets are presented which cause the error MSE surface, when either the sigmoid or the ReLu activation function is used, to have a suboptimal local minimum.”