Empirically Characterizing Overparameterization Impact on Convergence

By Newsha Ardalani, Joel Hestness, G. Diamos. 2018

4 citations recorded by the source corpus.

Subfields: Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

1. Eric Wallace
2. Kurt Keutzer
3. Sheng Shen
4. Daniel Klein
5. Joseph E. Gonzalez

Citations in the corpus

• Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers (2020, background): “Ardalani et al. (2019) show that larger RNN models take fewer gradient iterations to converge but do not consider that larger models are faster when adjusting for wall-clock time.”