A Diffusion Theory For Minima Selection: Stochastic Gradient Descent Escapes Sharp Minima Exponentially Fast

By Issei Sato, Masashi Sugiyama. 2020

6 citations recorded by the source corpus.

Subfields: Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Jascha Narain Sohl-Dickstein
2. Yasaman Bahri
3. Guy Gur-Ari
4. Ethan Dyer
5. Greg Yang
6. Sam Schoenholz
7. Yann Dauphin
8. Roger Baker Grosse

Citations in the corpus

• The large learning rate phase of deep learning: the catapult mechanism (2020, background): “SGD training with large initial learning rates often leads to improved performance over training with small initial learning rates (see Li et al. (2019); Leclerc & Madry (2020); Xie et al. (2020); Frankle et al. (2020); Jastrzebski et al. (2020) for recent discussions).”