Optimal SGD Hyperparameters for Fully Connected Networks

By Daniel S. Park. 2018

Subfields: Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Greg Yang
2. Sam Schoenholz
3. Jeffrey Pennington
4. Lechao Xiao
5. Jaehoon Lee
6. Roman Novak
7. Jascha Narain Sohl-Dickstein
8. Yasaman Bahri

Citations in the corpus

• Wide neural networks of any depth evolve as linear models under gradient descent (2019, background): “It has already been adopted in several recent works (van Laarhoven, 2017; Karras et al., 2018; Jacot et al., 2018; Du et al., 2018; Park et al., 2018).”
• Freeze and Chaos for DNNs: an NTK view of Batch Normalization, Checkerboard and Boundary Effects (2019, background): “[21] Park, Daniel S, Smith, Samuel L, Sohl-dickstein, Jascha, & Le, Quoc V.”