Exploring Generalization in Deep Learning

By Behnam Neyshabur, Srinadh Bhojanapalli, David A. McAllester, Nati Srebro. 2017

1,437 citations recorded by the source corpus.

With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

Subfields: Artificial Intelligence, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Quoc V. Le
  3. Percy Liang
  4. Sergey Levine
  5. Ruslan Salakhutdinov
  6. Mohit Bansal
  7. Dacheng Tao
  8. Jianfeng Gao

In Scaling & Learning Theory

  1. Tengyu Ma
  2. Yuanzhi Li
  3. Jason D. Lee
  4. Samet Oymak
  5. Colin Wei
  6. Behnam Neyshabur
  7. Quanquan Gu
  8. Simon Shaolei Du

Citations in the corpus