Measuring the tendency of CNNs to Learn Surface Statistical Regularities

By Jason Jo, Yoshua Bengio. 2017

270 citations recorded by the source corpus.

Deep CNNs are known to exhibit the following peculiarity: on the one hand they generalize extremely well to a test set, while on the other hand they are extremely sensitive to so-called adversarial perturbations. The extreme sensitivity of high performance CNNs to adversarial examples casts serious doubt that these networks are learning high level abstractions in the dataset. We are concerned with the following question: How can a deep CNN that does not learn any high level semantics of the dataset manage to generalize so well? The goal of this article is to measure the tendency of CNNs to learn surface statistical regularities of the dataset. To this end, we use Fourier filtering to construct datasets which share the exact same high level abstractions but exhibit qualitatively different surface statistical regularities. For the SVHN and CIFAR-10 datasets, we present two Fourier filtered variants: a low frequency variant and a randomly filtered variant. Each of the Fourier filtering schemes is tuned to preserve the recognizability of the objects. Our main finding is that CNNs exhibit a tendency to latch onto the Fourier image statistics of the training dataset, sometimes exhibiting up to a 28% generalization gap across the various test sets. Moreover, we observe that significantly increasing the depth of a network has a very marginal impact on closing the aforementioned generalization gap. Thus we provide quantitative evidence supporting the hypothesis that deep CNNs tend to learn surface statistical regularities in the dataset rather than higher-level abstract concepts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Probing Neural Network Comprehension of Natural Language Arguments (2019, background): “There is much recent work focusing on statistical cues in datasets in vision (Jo and Bengio, 2017) and NLP (Sanchez et al., 2018; McCoy et al., 2019; Gururangan et al., 2018; Glockner et al., 2018; Poliak et al., 2018; Rajpurkar et al., 2018; Jia and Liang, 2017).”
Shortcut learning in deep neural networks (2020, background): “In addition to shortcut opportunities that are fairly easy to recognise, deep learning has led to the discovery of much more subtle shortcut features, including high-frequency patterns that are almost invisible to the human eye [32, 33].”
Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “Indeed, recent papers have attested to the tendency of neural networks to rely on surface statistical regularities rather than learning global concepts (Jo and Bengio, 2017; Geirhos et al., 2019).”
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data (2019, background): “For example, in computer vision, researchers have found that deep neural networks rely on surface-level texture [12, 5] or clues in the image’s background to recognize foreground objects even when that seems both unnecessary and somehow wrong: the beach is not what makes a seagull a seagull.”
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “We show that excessive sensitivity and invariance can be tied respectively to the existence of generalizable non-robust features (Jo & Bengio, 2017; Ilyas et al., 2019; Yin et al., 2019) and to robust features that are predictive for standard datasets, but not for the general vision tasks that…”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “Rather, we hypothesize that current ML models are highly vulnerable to adversarial examples because they learn superficial data statistics [19, 13, 18] that share some properties of these simple distributions.”
On the Binding Problem in Artificial Neural Networks (2020, background): “In general, evidence indicates that neural networks learn mostly about surface statistics (e.g. between textures and classifications in images) in place of the underlying concepts (Jo and Bengio, 2017; Karpathy et al., 2015; Lake and Baroni, 2018).”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, methodology): “We followed the radial and random Fourier ﬁltering introduced in (Jo & Bengio, 2017) to attach the surface statistical regularities into the images in MNIST.”
Excessive Invariance Causes Adversarial Vulnerability (2018, background): “The difference to human perception revealed by adversarial examples fundamentally questions which statistics the network use to base its decisions (Jo & Bengio, 2017; Tsipras et al., 2018).”
A Fourier Perspective on Model Robustness in Computer Vision (2019, background): “While this problem is far from being completely understood, perhaps the simplest explanation proposed is that models lack robustness to distributional shift simply because there is no reason for them to be robust (Jo & Bengio, 2017).”
Measuring abstract reasoning in neural networks (2018, background): “This potential pitfall is even more acute in the case of neural networks, given their striking capacity for memorization (Zhang et al., 2016) and ability to exploit superﬁcial statistical cues (Jo & Bengio, 2017; Szegedy et al., 2013).”
Semantic Adversarial Examples (2018, methodology): “In [17], Fourier ﬁltering methods are used to slightly modify images such that they contain the same high level abstractions but exhibit different surface statistical regularities.”
Adversarial Examples on Object Recognition (2020, background): “ate adversarial inputs which are not intelligible by humans (as in [126]) shows that DNNs use diﬀerent features than we wish for. ResearchinadversarialexamplesstrengthentheconclusionsfromJo and Bengio[88]whichanalyzed convolutional networks in diﬀerent regimes and showed they exhibit a tendency to l”
Deep Repulsive Prototypes for Adversarial Robustness (2021, background): “Previously, Jo and Bengio (2017) showed that neural networks have a tendency to learn surface regularities rather than higher-level abstractions.”
Explaining The Efficacy of Counterfactually-Augmented Data (2020, background): “Of particular concern, these models have been shown to rely on spurious signals, such as surface-level textures in images (Jo & Bengio, 2017; Geirhos et al., 2018), and background scenery—even when the task is to recognize foreground objects (Beery et al., 2018).”
Neural network models and deep learning - a primer for biologists (2019, background): “…Yamins et al., 2014; Güçlü and van Gerven, 2015; Cichy et al., 2016; Eickenberg et al., 2017; Wen et al., 2017; Nayebi et al., 2018), and yet multiple functional incompatibilities have already been reported (Szegedy et al., 2013; Geirhos et al., 2017; Jo and Bengio, 2017; Rajalingham et al., 2018).”
Location Attention for Extrapolation to Longer Sequences (2019, background): “As neural networks are powerful memorizers (Zhang et al., 2017) and easily learn superﬁcial ∗ Shared senior authorship statistical cues (Jo and Bengio, 2017), testing extrapolation and generalization to samples from the long tails of a distribution might be the only way of quantifying their…”