Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning

By Hongge Chen, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Cho-Jui Hsieh. 2017

137 citations recorded by the source corpus.

Visual language grounding is widely studied in modern neural image captioning systems, which typically adopts an encoder-decoder framework consisting of two principal components: a convolutional neural network (CNN) for image feature extraction and a recurrent neural network (RNN) for language caption generation. To study the robustness of language grounding to adversarial perturbations in machine vision and perception, we propose Show-and-Fool, a novel algorithm for crafting adversarial examples in neural image captioning. The proposed algorithm provides two evaluation approaches, which check if we can mislead neural image captioning systems to output some randomly chosen captions or keywords. Our extensive experiments show that our algorithm can successfully craft visually-similar adversarial examples with randomly targeted captions or keywords, and the adversarial examples can be made highly transferable to other image captioning systems. Consequently, our approach leads to new robustness implications of neural image captioning and novel insights in visual language grounding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, background): “[2, 16] studied how to craft adversarial examples for image captioning, and [10] investigated how to derive adversarial rules to attack VQA systems.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “A few targeted attacks include Liang et al. (2018), which speciﬁed a desired class to fool a text classiﬁer, and Chen et al. (2018a), which speciﬁed words or captions to generate in an image captioning model.”
PAWS: Paraphrase Adversaries from Word Scrambling (2019, background): “Example perturbation rules include adding noise to inputs (Jia and Liang, 2017; Chen et al., 2018),”
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification (2019, methodology): “question answering (Jia and Liang, 2017; Chen et al., 2018; Ribeiro et al., 2018), textual entailment (Zhao et al.”
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models (2020, background): “First, unlike many existing studies in other domains such as images (Goodfellow et al., 2014; Chen, 2018), here the input search space is discrete and thus the traditional gradient-based optimization methodologies cannot be harnessed effectively (Zhang et al.”
A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captions (2019, background): “is known to be difficult and sensitive to input perturbations (Chen et al., 2018).”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “DeepFool is the algorithm that computes ρ in (10) to compute the robustness defined by (11).”
Adversarial Robustness vs. Model Compression, or Both? (2019, background): “, structure and weight parameters) [7, 8, 50, 44]; or black-box accesses where the adversary can only make queries and observe outputs [9, 22].”
On Evaluating Adversarial Robustness of Large Vision-Language Models (2023, background): “…are devoted to investigating adversarial attacks against visual question answering [6, 11, 37, 38, 43, 77, 89, 97, 105] and image caption [1, 14, 99], with the majority of these efforts focusing on conventional CNN-RNN-based models, assuming white-box access or untargeted adversarial goals,…”
Is Robustness the Cost of Accuracy? - A Comprehensive Study on the Robustness of 18 Deep Image Classification Models (2018, background): “A common approach to evaluate the robustness of DNNs is via adversarial attacks [3,4,5,6,7,8,9,10,11], where imperceptible adversarial examples are crafted to mislead DNNs.”
Towards Stable and Efficient Training of Verifiably Robust Neural Networks (2019, background): “The robustness of DNNs under adversarial examples is well-studied from both attack (crafting powerful adversarial examples) and defence (making the model more robust) perspectives (Athalye et al., 2018; Carlini & Wagner, 2017a;b; Goodfellow et al., 2015; Madry et al., 2018; Papernot et al., 2016; Xi”
Say What I Want: Towards the Dark Side of Neural Dialogue Models (2019, background): “First, unlike similar works such as [22], where the authors try to craft inputs for a neural image captioning model to output targeted sentences, our problem involves discrete inputs (i.”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “Image Captioning.”
Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach (2018, background): “It has been observed recently that machine learning algorithms, especially deep neural networks, are vulnerable to adversarial examples [3, 4, 5, 6, 7, 8].”
Exact Adversarial Attack to Image Captioning via Structured Output Learning With Latent Variables (2019, methodology): “metric Show-and-Fool [6] Our methods max logits max margin of logits GEM latent SSVMs ‖ǫ‖2 ↓ 1.”