L ] 1 0 A pr 2 01 9 AdaGrad-Norm convergence over nonconvex landscapes AdaGrad stepsizes : sharp convergence over nonconvex landscapes , from any initialization

By Rachel A. Ward, Xiaoxia Wu, L. Eon Bottou. 2019

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Zachary B. Charles
2. Jakub Konecný
3. Sashank J. Reddi
4. Manzil Zaheer
5. H. Brendan McMahan
6. Sai Praneeth Karimireddy
7. Sanjiv Kumar
8. Fangyu Zou

Citations in the corpus

• Adaptive Federated Optimization (2020, background): “, 2011; Kingma and Ba, 2015) and nonconvex setting (Zaheer et al., 2018; Li and Orabona, 2018; Ward et al., 2018; Wu et al., 2019).”
• A Sufficient Condition for Convergences of Adam and RMSProp (2018, background): “Besides, many other adaptive stochastic algorithms without combining momentums, such as AdaGrad [22, 18] and stagewise AdaGrad [7], have been guaranteed to be convergent and work well in the non-convex stochastic setting.”