Can you fool AI with adversarial examples on a visual Turing test?

By Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, Dawn Song. 2017

42 citations recorded by the source corpus.

Deep learning has achieved impressive results in many areas of Computer Vision and Natural Language Pro- cessing. Among others, Visual Question Answering (VQA), also referred to a visual Turing test, is considered one of the most compelling problems, and recent deep learning models have reported significant progress in vision and language modeling. Although Artificial Intelligence (AI) is getting closer to passing the visual Turing test, at the same time the existence of adversarial examples to deep learning systems may hinder the practical application of such systems. In this work, we conduct the first extensive study on adversarial examples for VQA systems. In particular, we focus on generating targeted adversarial examples for a VQA system while the target is considered to be a question-answer pair. Our evaluation shows that the success rate of whether a targeted adversarial example can be generated is mostly dependent on the choice of the target question-answer pair, and less on the choice of images to which the question refers. We also report the language prior phenomenon of a VQA model, which can explain why targeted adversarial examples are hard to generate for some question-answer targets. We also demonstrate that a compositional VQA architecture is slightly more resilient to adversarial attacks than a non-compositional one. Our study sheds new light on how to build deep vision and language resilient models robust against adversarial examples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, background): “In addition to image classification, attacks to other DNN-powered tasks have also been actively investigated, such as visual question answering [43, 1], image captioning [10], semantic segmentation [31, 1] and etc [12, 8, 39].”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [19, 49], question answering [26], voice recognition [8, 9, 47, 60], and other domains (for instance, see [1, 3, 12, 20, 23, 24, 30, 58]).”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[40] generated adversarial attacks for the Visual Turing Test, also known as ‘Visual Question Answer’ (VQA).”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, background): “attacks to other CNN-related tasks have also been actively investigated. Typical examples include semantic segmentation and object detection [21, 22, 23, 24, 25], image captioning [26], and visual QA [27]. Besides, recent studies have also conducted adversarial attacks in the real-world scenarios, i”
Review of Artificial Intelligence Adversarial Attack and Defense Technologies (2019, methodology): “[84] launched an adversarial attack for Visual Turing test, known as Visual Question Answer (VQA).”
Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss (2019, unknown): “Deep neural networks (DNNs) have been widely used for security-critical tasks, including but not limited to autonomous driving [Evtimov et al. , 2017], surveillance [Ouyang and Wang, 2013], biometric recognition [Xu et al. , 2017], and malware detection [Yuan et al. , 2014].”
Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving (2019, methodology): “Further, optimization-based algorithms have been leveraged to generate adversarial examples on various kinds of machine learning tasks successfully, such as segmentation [26, 53], human pose estimation [26], object detection [53], Visual Question Answer system [54],”
On the Robustness of Semantic Segmentation Models to Adversarial Attacks (2017, background): “(6) The aforementioned attacks were all proposed in the context of image classification, but they have been adapted to the problems of semantic segmentation [26, 16], object detection [66] and visual question answering [68].”