Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data

By Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-ling Wang, Michael I. Jordan. 2018

122 citations recorded by the source corpus.

We present a probabilistic framework for studying adversarial attacks on discrete data. Based on this framework, we derive a perturbation-based method, Greedy Attack, and a scalable learning-based method, Gumbel Attack, that illustrate various tradeoffs in the design of attacks. We demonstrate the effectiveness of these methods using both quantitative metrics and human evaluation on various state-of-the-art models for text classification, including a word-based CNN, a character-based CNN and an LSTM. As as example of our results, we show that the accuracy of character-based convolutional networks drops to the level of random selection by modifying only five characters through Greedy Attack.

Subfields: Artificial Intelligence, Sociology and Political Science, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “The alignments obtained via different attention mechanisms have produced visualizations ranging from tasks like NLI (Rocktäschel et al., 2016; Yin et al., 2016), summarization (Rush et al., 2015), MT postediting (Jauregi Unanue et al., 2018), and morphological inflection (Aharoni and Goldberg,…”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, methodology): “Yang et al.(2018) applied a greedy approach and a Gumbel trick to speed up the inference time.”
Learning to Explain: An Information-Theoretic Perspective on Model Interpretation (2018, background): “The explainer of L2X is composed of a global component and component (See Figure 2 in Yang et al. (2018)).”
Universal Adversarial Attacks on Text Classifiers (2019, background): “[19] proposed a probabilistic framework for maximizing the success probability of an adversarial attack.”
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency (2019, methodology): “Yang et al. (2018) introduce a perturbation-based method called Greedy Attack and a scalable learning-based method called Gumbel Attack.”
On the Robustness of Self-Attentive Models (2019, methodology): “The third method (denoted as GS-GR ) greed-ily searches for the weak spot of the input sentence (Yang et al., 2018) by replacing each word, one at a time, with a “padding” (a zero-valued vector) and examining the changes of output probability.”
BERT-ATTACK: Adversarial Attack against BERT Using BERT (2020, methodology): “Previous methods craft adversarial samples mainly based on speciﬁc rules (Li et al., 2018; Gao et al., 2018; Yang et al., 2018; Jin et al., 2019).”
Adversarial Reprogramming of Text Classification Neural Networks (2018, methodology): “There has been a similar line of adversarial attacks on neural networks with discrete input domains (Papernot et al., 2016c; Yang et al., 2018), where the adversary modiﬁes a few tokens in the input sequence to cause mis-classiﬁcation by a sequence model.”