Revisiting the Generalization of Adaptive Gradient Methods

By Naman Agarwal, Rohan Anil, Elad Hazan, Tomer Koren, Cyril Zhang. 2019

5 citations recorded by the source corpus.

Subfields: Optimization & Training, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Ariel Kleiner
2. Pierre Foret
3. Behnam Neyshabur
4. Hossein Mobahi
5. Nati Srebro
6. Jason D. Lee
7. Ping Li
8. Anshumali Shrivastava

In Scaling & Learning Theory

1. Hossein Mobahi
2. Behnam Neyshabur
3. Tengyu Ma
4. Colin Wei
5. J. Kolter
6. Vaishnavh Nagarajan
7. Hanie Sedghi
8. Samet Oymak

Citations in the corpus

• Sharpness-Aware Minimization for Efficiently Improving Generalization (2020, background): “, 2011; Dozat, 2016; Martens and Grosse, 2015)) has become an important design choice, though understanding of its relationship to model generalization remains nascent (Shirish Keskar et al., 2016; Wilson et al., 2017; Shirish Keskar and Socher, 2017; Agarwal et al., 2020; Jacot et al., 2018).”