Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency

By Shuhuai Ren, Yihe Deng, Kun He, Wanxiang Che. Annual Meeting of the Association for Computational Linguistics, 2019

822 citations recorded by the source corpus.

We address the problem of adversarial attacks on text classification, which is rarely studied comparing to attacks on image classification. The challenge of this task is to generate adversarial examples that maintain lexical correctness, grammatical correctness and semantic similarity. Based on the synonyms substitution strategy, we introduce a new word replacement order determined by both the word saliency and the classification probability, and propose a greedy algorithm called probability weighted word saliency (PWWS) for text adversarial attack. Experiments on three popular datasets using convolutional as well as LSTM models show that PWWS reduces the classification accuracy to the most extent, and keeps a very low word substitution rate. A human evaluation study shows that our generated adversarial examples maintain the semantic similarity well and are hard for humans to perceive. Performing adversarial training using our perturbed datasets improves the robustness of the models. At last, our method also exhibits a good transferability on the generated adversarial examples.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/1adfa30bf112de20cb959014e44626d760aa8e4e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

BERT-ATTACK: Adversarial Attack against BERT Using BERT (2020, background): “Adversarial Attack for Text Current successful attacks for text usually adopt heuristic rules to modify the characters of a word (Jin et al., 2019), and substituting words with synonyms (Ren et al., 2019).”
Explaining NLP Models via Minimal Contrastive Editing (MiCE) (2020, background): “, 2018) or word replacement (Alzantot et al., 2018; Ren et al., 2019; Garg and Ramakrishnan, 2020), cannot be used to generate contrastive edits.”
Contextualized Perturbation for Textual Adversarial Attack (2020, background): “First, it allows attacking any position of the input sequence, whereas existing synonym replacement approaches can generally only attack tokens in a predeﬁned vocabulary (Alzantot et al., 2018; Jin et al., 2020; Ren et al., 2019, inter alia ).”
The rise and potential of large language model based agents: a survey (2025, neutral): “Researchers have found that pre-trained language models (PLMs) are particularly susceptible to adversarial attacks, leading to erroneous answers [507,516,517].”
BAE: BERT-based Adversarial Examples for Text Classification (2020, methodology): “We estimate token importance I i of each t i ∈ S , by deleting t i from S and computing the decrease in probability of predicting the correct label y , similar to Jin et al. (2019); Ren et al. (2019).”
A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement (2022, background): “[Ren et al., 2019; Li et al., 2020; Wang et al., 2021] adopt the adversarial examples generated by their attack methods for adversarial training and achieve robustness improvement.”
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT (2020, background): “Existing approaches generate malicious adversarial examples as attacks by ﬁnding the minimum perturbation on each sample to study the robustness of a model (Liang et al., 2017; Ebrahimi et al., 2017; Gao et al., 2018; Li et al., 2018; Alzantot et al., 2018; Ren et al., 2019).”
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP (2020, background): “(Ren et al., 2019) Untargeted Classification WordNet-based synonym swap Greedy-WIR (saliency)”
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, background): “Textual adversarial attacks typically perform word-level substitution (Ebrahimi et al., 2018; Alzantot et al., 2018; Ren et al., 2019) or sentence-level paraphrasing (Iyyer et al.”
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization (2021, methodology): “To create multiple positive samples, in our pilot study, we experiment with paraphrasing with synonym substitution (Ren et al., 2019), randomly replacing words based on the prediction of masked language models (Kobayashi, 2018), and back-translation (Mallinson et al., 2017).”
On Robust Prefix-Tuning for Text Classification (2022, methodology): “While we calculate the layerwise canonical manifolds with the correctly classiﬁed samples from both the clean training set and the PWWS-perturbed training set, it can be seen that the robustness against PWWS attack is still enhanced by several percent, while the clean accuracy is slightly affected.”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “[92] also used SNLI dataset.”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, neutral): “The threat models employed in this literature include synonym-based attacks [62–64], character-based substitutions [24], and spelling mistakes [65].”
Evaluating the Robustness of Neural Language Models to Input Perturbations (2021, methodology): “However, Negation was included in CheckList, and Replacement with Synonyms was used for adversarial attack (Dong et al., 2020; Ren et al., 2019).”
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning (2020, methodology): “We use PWWS (Ren et al., 2019) and TextFooler (Jin et al., 2020) as our attack methods, which have been shown to effectively attack state-of-the-art NLP models including PLMs like BERT.”
Can Large Language Models Be an Alternative to Human Evaluations? (2023, methodology): “As mentioned in Section 4.3, one teacher rates the fluency of Textfooler significantly higher than PWWS while the other two teachers do not.”
Gradient-based Adversarial Attacks against Text Transformers (2021, methodology): “ones with similar word embeddings (Alzantot et al., 2018; Ren et al., 2019; Jin et al., 2020).”
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces (2020, background): “So replacing characters (Ebrahimi et al., 2017), words (Alzantot et al., 2018; Jin et al., 2019; Ren et al., 2019; Papernot et al., 2016) or paraphrasing sentences (Jia and Liang, 2017) are the main streams of generating adversarial samples in texts.”
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, methodology): “For example, by enumerating feasible substitutions on the basis of word similarity and language models, Ren et al. (2019) and Garg and Ramakr-ishnan (2020) select adversarial word replacements which severely inﬂuence the predictions from the text classiﬁcation model.”