Towards Crafting Text Adversarial Samples

By Suranjana Samanta, Sameep Mehta. 2017

230 citations recorded by the source corpus.

Adversarial samples are strategically modified samples, which are crafted with the purpose of fooling a classifier at hand. An attacker introduces specially crafted adversarial samples to a deployed classifier, which are being mis-classified by the classifier. However, the samples are perceived to be drawn from entirely different classes and thus it becomes hard to detect the adversarial samples. Most of the prior works have been focused on synthesizing adversarial samples in the image domain. In this paper, we propose a new method of crafting adversarial text samples by modification of the original samples. Modifications of the original text samples are done by deleting or replacing the important or salient words in the text or by introducing new words in the text sample. Our algorithm works best for the datasets which have sub-categories within each of the classes of examples. While crafting adversarial samples, one of the key constraint is to generate meaningful sentences which can at pass off as legitimate from language (English) viewpoint. Experimental results on IMDB movie review dataset for sentiment analysis and Twitter dataset for gender detection show the efficiency of our proposed method.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “Others computed gradients with respect to input word embeddings to identify and rank words to be modiﬁed (Samanta and Mehta, 2017; Liang et al., 2018).”
Negative Training for Neural Dialogue Response Generation (2019, neutral): “sentence classification, or seq2seq models) (Papernot et al., 2016; Samanta and Mehta, 2017; Liang et al., 2018; Ebrahimi et al., 2017; Belinkov and Bisk, 2017; Chen et al., 2017).”
Detecting egregious responses in neural sequence-to-sequence models (2018, background): “So a set of (usually hand-crafted) rules (Belinkov & Bisk, 2017; Samanta & Mehta, 2017; Jia & Liang, 2017) needs to be used to constrain the crafting process of adversarial examples.”
Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning (2020, background): “Dozens of attacks and defenses have been proposed [4, 10, 22, 29, 34, 46, 62, 63, 72, 73].”
TextBugger: Generating Adversarial Text Against Real-world Applications (2018, background): “In [33], Samanta et al. used the embedding gradient to determine important words.”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “A few white-box attacks look for important text edit operations that will fool a classifier [99, 207, 268, 294].”
Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers (2018, methodology): “Recently, [25] used the embedding gradient to determine important words.”
Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data (2018, background): “The same metric was used [5, 22].”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “The works (Samanta & Mehta, 2017; Iyyer et al., 2018) start to craft adversarial sentences that grammatically correct and maintain the syntax structure of the original sentence.”
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency (2019, background): “Samanta and Mehta (2017) propose to build a candidate pool that includes synonyms, typos and genre specific keywords.”
Gradient-based Adversarial Attacks against Text Transformers (2021, background): “Attacks that operate on the word level adopt heuristics such as synonym substitution (Samanta and Mehta, 2017; Zang et al., 2020; Maheshwary et al., 2020) or replacing words by”
Interpretable Adversarial Perturbation in Input Embedding Space for Text (2018, methodology): “Moreover, a method generated a large number of input sentences by re-placing a word with its synonym [ Samanta and Mehta, 2017 ] The primary strategy for generating adversarial examples in the NLP ﬁeld clearly differs from those developed in the image processing ﬁeld, which are rather ad-hoc, e.g.,…”
Contextualized Perturbation for Textual Adversarial Attack (2020, background): “Recent efforts have explored heuristic rules, such as replacing tokens with their synonyms (Samanta and Mehta, 2017; Super ant colony hits Australia {Coast, , } .”
Universal Adversarial Attacks on Text Classifiers (2019, methodology): “In [16, 17, 18], the gradient of the loss with respect to the inputs is used for discovering the critical words or characters in each input sample, and then perturbing them.”