Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects

By Michael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, Anh Totti Nguyen. 2018

329 citations recorded by the source corpus.

Despite excellent performance on stationary test sets, deep neural networks (DNNs) can fail to generalize to out-of-distribution (OoD) inputs, including natural, non-adversarial ones, which are common in real-world settings. In this paper, we present a framework for discovering DNN failures that harnesses 3D renderers and 3D models. That is, we estimate the parameters of a 3D renderer that cause a target DNN to misbehave in response to the rendered image. Using our framework and a self-assembled dataset of 3D objects, we investigate the vulnerability of DNNs to OoD poses of well-known objects in ImageNet. For objects that are readily recognized by DNNs in their canonical poses, DNNs incorrectly classify 97% of their pose space. In addition, DNNs are highly sensitive to slight pose perturbations. Importantly, adversarial poses transfer across models and datasets. We find that 99.9% and 99.4% of the poses misclassified by Inception-v3 also transfer to the AlexNet and ResNet-50 image classifiers trained on the same ImageNet dataset, respectively, and 75.5% transfer to the YOLOv3 object detector trained on MS COCO.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “However, research in the subsequent years has repeatedly found that these models still make many simple mistakes (Dodge & Karam, 2017; Geirhos et al., 2018; Alcorn et al., 2019), and new benchmarks testing these systems has often found their performance to be much lower than both their ImageNet accu”
Shortcut learning in deep neural networks (2020, background): “formed. To DNNs, in contrast, innocuous transformations can completely change predictions. This has been reported in various cases such as shifting the image by a few pixels [47], rotating the object [49], adding a bit of random noise or blur [51,50,52,53] or (as discussed earlier) by changing backg”
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence (2020, background): “ome sentences correctly but fails in the presence of unrelated distractor material. Sample of how an object in a noncanonical orientation and context fools many current object classification systems (Alcorn et al., 2018) Sample of how adversarially inserted material fools a large-scale language mode”
Robust Speech Recognition via Large-Scale Weak Supervision (2022, background): “…works have shown and continually reiterated how despite high performance on IID test sets, machine learning models can still make many mistakes when evaluated in even slightly different settings (Lake et al., 2017; Jia & Liang, 2017; Alcorn et al., 2019; Barbu et al., 2019; Recht et al., 2019).”
Adversarial Filters of Dataset Biases (2020, positive): “These results echo other ﬁndings that suggest that common biases that naturally occur in web-scale image data, such as towards canonical poses (Alcorn et al., 2019) or towards texture rather than shape (Geirhos et al., 2018), are problems for ImageNet-trained classiﬁers.”
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, background): “Examples include the failure to generalize to images with uncommon poses of objects [Alcorn et al., 2019] or to cope with small distributional changes [e.g. Zech et al., 2018, Touvron et al., 2019].”
In Search of Lost Domain Generalization (2020, background): “For instance, self-driving car systems struggle to perform under conditions different to those of training, including variations in light [Dai and Van Gool, 2018], weather [Volk et al., 2019], and object poses [Alcorn et al., 2019].”
Robust fine-tuning of zero-shot models (2021, background): “…model behavior under synthetic (Hendrycks & Dietterich, 2019; Tram`er et al., 2017; Madry et al., 2017; Geirhos et al., 2018b; Eykholt et al., 2018; Alcorn et al., 2019) and natural distribution shift (Hendrycks et al., 2021a; Koh et al., 2021; Wang et al., 2019; Barbu et al., 2019; Hendrycks et…”
Challenges and Prospects in Vision and Language Research (2019, background): “This can enable algorithms to perform well for the wrong reasons and algorithms can end up catastrophically failing in uncommon scenarios (Agrawal et al., 2018; Alcorn et al., 2019).”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “Although their performance in several of these tasks approaches that of humans, recent ﬁndings show that CNNs differ in intriguing ways from human vision, indicating fundamental deﬁciencies in our understanding of these models [87, 70, 32, 23, 76, 37, 3, 44, 46, 1, 48].”
Counterfactual Generative Networks (2021, background): “Despite the considerable successes of DNNs, they still struggle in many situations, e.g., classifying images perturbed by an adversary (Szegedy et al., 2013), or failing to recognize known objects in unfamiliar contexts (Rosenfeld et al., 2018) or from unseen poses (Alcorn et al., 2019).”
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations (2022, background): “Multiple works demonstrated that natural image datasets contain spurious correlations that hurt neural network models [50, 100, 89, 3, 90, 91, 64].”
Partial success in closing the gap between human and machine vision (2021, background): “Many OOD generalisation tests have been proposed: ImageNet-C [15] for corrupted imaes, ImageNet-Sketch [16] for sketches, Stylized-ImageNet [17] for image style changes, [18] for unfamiliar object poses, and many more [19–29].”
Detecting semantic anomalies (2019, background): “, 2019), canonical pose (Alcorn et al., 2019), or contextual cues (Beery, Horn, and Perona, 2018) rather than learning globally coherent characteristics of objects.”
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization (2021, background): “Accuracy suffers when the model performs under conditions different to those of training, such as variations in light [8], weather [51], object poses [2], textures [16], or object backgrounds [5].”
Language Models are Unsupervised Multitask Learners (2019, background): “, 2017), reading comprehension systems (Jia & Liang, 2017), and image classifiers (Alcorn et al., 2018) on the diversity and variety of possible inputs highlights some of the shortcomings of this approach.”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, background): “g.[11, 13, 34, 20, 14, 2, 10]) that worst-case prediction performance drops dramatically for neural network classifiers obtained using standard training, even for rather small transformation sets.”