Assessing Shape Bias Property of Convolutional Neural Networks

By Hossein Hosseini, Baicen Xiao, Mayoore S. Jaiswal, Radha Poovendran. 2018

40 citations recorded by the source corpus.

It is known that humans display "shape bias" when classifying new items, i.e., they prefer to categorize objects based on their shape rather than color. Convolutional Neural Networks (CNNs) are also designed to take into account the spatial structure of image data. In fact, experiments on image datasets, consisting of triples of a probe image, a shape-match and a color-match, have shown that one-shot learning models display shape bias as well. In this paper, we examine the shape bias property of CNNs. In order to conduct large scale experiments, we propose using the model accuracy on images with reversed brightness as a metric to evaluate the shape bias property. Such images, called negative images, contain objects that have the same shape as original images, but with different colors. Through extensive systematic experiments, we investigate the role of different factors, such as training data, model architecture, initialization and regularization techniques, on the shape bias property of CNNs. We show that it is possible to design different CNNs that achieve similar accuracy on original images, but perform significantly different on negative images, suggesting that CNNs do not intrinsically display shape bias. We then show that CNNs are able to learn and generalize the structures, when the model is properly initialized or data is properly augmented, and if batch normalization is used.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness (2018, background): “Ritter et al. (2017) discovered that CNNs develop a so-called “shape bias” just like children, i.e. that object shape is more important than colour for object classiﬁcation (although see Hosseini et al. (2018) for contrary evidence).”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, background): “It has been shown that CNNs trained on ImageNet are strongly biased towards recognizing texture [16], although other work shows that CNNs can be biased towards shape [33] or colour [20].”
Reducing Domain Gap by Reducing Style Bias (2021, background): “Others have attempted to mitigate the style bias of CNNs under the assumption that it deteriorates CNNs’ generalization capability [23, 14, 21].”
Reducing Domain Gap via Style-Agnostic Networks (2019, background): “1), we observe an explicit correlation between the bias and domain gap: shape-biased representation generalizes better across domains, which confirms the common intuition [23,14,21].”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “Hosseini et al. [47] found that CNNs can learn to generalize to negative (reversed-brightness) images for a held-out class if the training data is augmented with negative images for the other classes.”