Sensitivity and Generalization in Neural Networks: an Empirical Study

By Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Narain Sohl-Dickstein. 2018

514 citations recorded by the source corpus.

In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and generalization through an extensive empirical exploration of two natural metrics of complexity related to sensitivity to input perturbations. Our experiments survey thousands of models with various fully-connected architectures, optimizers, and other hyper-parameters, as well as four different image classification datasets. We find that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the norm of the input-output Jacobian of the network, and that it correlates well with generalization. We further establish that factors associated with poor generalization $-$ such as full-batch training or using random labels $-$ correspond to lower robustness, while factors associated with good generalization $-$ such as data augmentation and ReLU non-linearities $-$ give rise to more robust functions. Finally, we demonstrate how the input-output Jacobian norm can be predictive of generalization at the level of individual test points.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Wide neural networks of any depth evolve as linear models under gradient descent (2019, background): “Aside from its theoretical simplicity, the infinite-width limit is also of practical interest as wider networks have been found to generalize better [5, 7, 9, 10, 11].”
There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average (2018, methodology): “Both the reduced Jacobian norm and flatness of solutions have been related to generalization in the literature (Sokolić et al., 2017; Novak et al., 2018; Chaudhari et al., ar X iv :1 80 6.”
Fantastic Generalization Measures and Where to Find Them (2019, background): “Novak et al. (2018) conducted large scale study of neural networks but they only looked at correlation of a few measures to generalization.”
Three Mechanisms of Weight Decay Regularization (2018, methodology): “We show that when K-FAC is applied to a linear network using the Gauss-Newton metric (K-FAC-G), weight decay is equivalent to regularizing the squared Frobenius norm of the input-output Jacobian (which was shown by Novak et al. (2018) to improve generalization).”
Is Generator Conditioning Causally Related to GAN Performance? (2018, background): “Novak et al. (2018) explore the relationship between the norm of that Jacobian and the generalization error of the classifier.”
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization (2019, background): “…(Dauphin et al., 2014; Choromanska et al., 2014; Goodfellow et al., 2014; Im et al., 2016; Wu et al., 2017; Liao & Poggio, 2017; Cooper, 2018; Novak et al., 2018), and to the implicit regularization properties of SGD (Brutzkus et al., 2017; Zhang et al., 2018a; Poggio et al., 2017), though a…”
Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks (2018, background): “Similar empirical observations have been made over a wide range of architectural and hyper-parameter choices [15, 24, 14].”
On the Spectral Bias of Neural Networks (2018, background): “…transform affords a natural way of measuring how fast a function can change within a small neighborhood in its input domain ; as such, it is a strong candidate for quantifying and analyzing the sensitivity of a model – which in turn provides a natural measure of complexity (Novak et al., 2018).”
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, positive): “However, as can be seen in Figures 1, 6 (c), 7, and Table 1, the best CNN overall outperforms the best CNN-GP by a significant margin – an observation specific to CNNs and not FCNs or LCNs.7 We observe this gap in performance especially in the case of ReLU networks trained with a large learning rate”
Learning Differential Equations that are Easy to Solve (2020, background): “Novak et al. (2018); Drucker & LeCun (1992) regularized the gradients of neural networks to improve generalization.”
The Nonlinearity Coefficient - Predicting Generalization in Deep Neural Networks (2018, background): “Other papers Finally, we want to point to two papers that conducted largescale studies similar to ours where test error was predicted across a variety of networks (Novak et al., 2018; Jiang et al., 2019).”
Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem (2018, background): “One very intriguing observation in this context is that overparameterized networks with ReLU activations, which are trained with gradient based methods, often exhibit better generalization error than smaller networks (Neyshabur et al., 2014; 2019; Novak et al., 2018).”
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes (2019, background): “[42] Roman Novak, Yasaman Bahri, Daniel A.”
Detecting Memorization in ReLU Networks (2018, background): “The empirical studies of Morcos et al. (2018) and Novak et al. (2018) explore robustness to specific types of noise.”
How to train your neural ODE (2020, unknown): “In the context of generalization, Jacobian regularization is analagous to gradient regularization, which has been shown to improve generalization (Drucker & LeCun, 1992; Novak et al., 2018).”