Show-and-Fool: Crafting Adversarial Examples for Neural Image Captioning

By Hongge Chen, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Cho-Jui Hsieh. 2017

50 citations recorded by the source corpus.

Modern neural image captioning systems typically adopt the encoder-decoder framework consisting of two principal components: a convolutional neural network (CNN) for image feature extraction and a recurrent neural network (RNN) for caption generation. Inspired by the robustness analysis of CNN-based image classifiers to adversarial perturbations, we propose \textbf{Show-and-Fool}, a novel algorithm for crafting adversarial examples in neural image captioning. Unlike image classification tasks with a finite set of class labels, finding visually-similar adversarial examples in an image captioning system is much more challenging since the space of possible captions in a captioning system is almost infinite. In this paper, we design three approaches for crafting adversarial examples in image captioning: (i) targeted caption method; (ii) targeted keyword method; and (iii) untargeted method. We formulate the process of finding adversarial perturbations as optimization problems and design novel loss functions for efficient search. Experimental results on the Show-and-Tell model and MSCOCO data set show that Show-and-Fool can successfully craft visually-similar adversarial examples with randomly targeted captions, and the adversarial examples can be made highly transferable to the Show-Attend-and-Tell model. Consequently, the presence of adversarial examples leads to new robustness implications of neural image captioning. To the best of our knowledge, this is the first work on crafting effective adversarial examples for image captioning tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, background): “In addition to image classification, attacks to other DNN-powered tasks have also been actively investigated, such as visual question answering [43, 1], image captioning [10], semantic segmentation [31, 1] and etc [12, 8, 39].”
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach (2018, methodology): “Adversarial examples can also be crafted from a black-box network model using an ensemble approach (Liu et al., 2016), training a substitute model (Papernot et al., 2017), or employing zeroth-order optimization based attacks (Chen et al., 2017c).”
Towards Fast Computation of Certified Robustness for ReLU Networks (2018, methodology): “Comparison of our proposed certified lower bounds Fast-Lin and Fast-Lip, LP and LP-Full, the estimated lower bounds by CLEVER, the exact minimum distortion by Reluplex, and the upper bounds by Attack algorithms (CW `∞ for p = ∞, CW `2 for p = 2, and EAD for p = 1) on 2, 3 layers toy MNIST networks w”
Negative Training for Neural Dialogue Response Generation (2019, background): “sentence classification, or seq2seq models) (Papernot et al., 2016; Samanta & Mehta, 2017; Liang et al., 2018; Ebrahimi et al., 2017; Belinkov & Bisk, 2017; Chen et al., 2017).”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, background): “Typical examples include semantic segmentation and object detection [21, 22, 23, 24, 25], image captioning [26], and visual QA [27].”
Detecting egregious responses in neural sequence-to-sequence models (2018, background): “There is a large body of work on adversarial attacks for deep learning models for the continuous input space, and most of them focus on computer vision tasks such as image classiﬁcation (Goodfellow et al., 2014; Szegedy et al., 2013) or image captioning (Chen et al., 2017).”
Efficient Neural Network Robustness Certification with General Activation Functions (2018, background): “For example, in image learning tasks such as object classification [3, 4, 5, 6] or content captioning [7], visually indistinguishable adversarial examples can be easily crafted from natural images to alter a NN’s prediction result.”
Sparse Adversarial Attack via Perturbation Factorization (2020, background): “It has been observed in many DNN based applications, such as image classification [14,9,21], image captioning [39,8], image retrieval [13,4], question answering [23], autonomous driving [24], automatic check-out [25], face recognition [12], face detection [22], etc.”
L2-Nonexpansive Neural Networks (2018, methodology): “…of networks including image classiﬁcation (Carlini & Wagner, 2017a; Goodfellow et al., 2014), speech recognition (Kreuk et al., 2018; Alzantot et al., 2018; Carlini & Wagner, 2018), image captioning (Chen et al., 2017) and natural language processing (Gao et al., 2018; Ebrahimi et al., 2017).”