On Adversarial Examples for Character-Level Neural Machine Translation

By Javid Ebrahimi, Daniel Lowd, Dejing Dou. 2018

233 citations recorded by the source corpus.

Evaluating on adversarial examples has become a standard procedure to measure robustness of deep learning models. Due to the difficulty of creating white-box adversarial examples for discrete text input, most analyses of the robustness of NLP models have been done through black-box adversarial examples. We investigate adversarial examples for character-level neural machine translation (NMT), and contrast black-box adversaries with a novel white-box adversary, which employs differentiable string-edit operations to rank adversarial changes. We propose two novel types of attacks which aim to remove or change a word in a translation, rather than simply break the NMT. We demonstrate that white-box adversarial examples are significantly stronger than their black-box counterparts in different attack scenarios, which show more serious vulnerabilities than previously known. In addition, after performing adversarial training, which takes only 3 times longer than regular training, we can improve the model’s robustness significantly.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Scaling Laws for Reward Model Overoptimization (2022, methodology): “Many works have demonstrated the existence of adversarial examples in all kinds of neural networks [Szegedy et al., 2013, Lin et al., 2017, Ebrahimi et al., 2018, Dai et al., 2018], and proposed methods to measure and increase neural network robustness [Gu and Rigazio, 2014, Zheng et al., 2016,…”
Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “Others targeted speciﬁc words to omit, replace, or include when attacking seq2seq models (Cheng et al., 2018; Ebrahimi et al., 2018a).”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Text classiﬁcation models have been shown to be sensitive to single input character change (Ebrahimi et al., 2018b) and ﬁrst-order logic inconsistencies (Minervini and Riedel, 2018).”
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020, background): “Many of these studies focus on augmenting the training data to improve robustness, especially with adversarial examples (Ebrahimi et al., 2018; Cheng et al., 2019; Karpukhin et al., 2019; Michel et al., 2019).”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, background): “Machine Translation [13, 24, 30, 156] TED Talks, WMT’16 Multimodal Translation Task”
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020, methodology): “For example, adversarial training (Goodfellow et al., 2015) can empirically improve the robustness of MT systems (Ebrahimi et al., 2018).”
Interpreting Predictions of NLP Models (2020, methodology): “We will also cover adversarial perturbations such as to-ken ﬂipping (Ebrahimi et al., 2018) and adding distractor sentences (Jia and Liang, 2017).”
Robust Neural Machine Translation with Doubly Adversarial Inputs (2019, methodology): “Following (Goodfellow et al., 2015; Miyato et al., 2017; Ebrahimi et al., 2018b), we study the white-box method to generate adversarial examples tightly guided by the training loss.”
Review of Artificial Intelligence Adversarial Attack and Defense Technologies (2019, methodology): “[75] proposed to use gradient estimation method to sort the adversarial operations and use a greedy search or beam search method to search the adversarial samples.”
Natural Perturbation for Robust Question Answering (2020, background): “A closely relevant line of work is adversarial perturbations to expose the weaknesses of systems upon local changes and criticize their lack robustness (Ebrahimi et al., 2018; Glockner et al., 2018).”
Universal Adversarial Triggers for NLP (2019, methodology): “Ebrahimi et al. (2018a) find similar for attacking neural machine translation.”
A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement (2022, background): “In recent years, a series of adversarial attack methods have been proposed, ranging from character-level word misspelling [ Ebrahimi et al. , 2018 ] , word-level substitution [ Zang et al. , 2020 ] to sentence-level paraphrasing [ Pei et al. , 2020 ] .”
Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering (2020, unknown): “In the NLP community, state-of-the-art textual DNN attackers [5,7,9] use a diﬀerent approach from those in the visual community to generate textual adversarial examples.”
Weight Poisoning Attacks on Pretrained Models (2020, background): “Deep neural models are well-known for being naturally susceptible to adversarial perturbations — carefully-chosen modiﬁcations of inputs that allow one to control the output of an otherwise well-trained model (Goodfellow et al., 2015; Ebrahimi et al., 2018b).”
Combating Adversarial Misspellings with Robust Word Recognition (2019, methodology): “Changes to text, whether word-or character-level, are all perceptible, raising some questions about what should rightly be considered an adversarial example (Ebrahimi et al., 2018b; Belinkov and Bisk, 2018).”