Combating Adversarial Misspellings with Robust Word Recognition

By Danish Pruthi, Bhuwan Dhingra, Zachary Chase Lipton. 2019

363 citations recorded by the source corpus.

To combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier. Our word recognition models build upon the RNN semi-character architecture, introducing several new backoff strategies for handling rare and unseen words. Trained to recognize words corrupted by random adds, drops, swaps, and keyboard mistakes, our method achieves 32% relative (and 3.3% absolute) error reduction over the vanilla semi-character model. Notably, our pipeline confers robustness on the downstream classifier, outperforming both adversarial training and off-the-shelf spell checkers. Against a BERT model fine-tuned for sentiment analysis, a single adversarially-chosen character attack lowers accuracy from 90.3% to 45.8%. Our defense restores accuracy to 75%. Surprisingly, better word recognition does not always entail greater robustness. Our analysis reveals that robustness also depends upon a quantity that we denote the sensitivity.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models (2021, background): “is relevant in making models more future-proof as well as more resilient to accidental or adversarial spelling mistakes (Pruthi et al., 2019; Sun et al., 2020).”
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text (2020, background): “Much previous work applies typing noise for language classiﬁer robustness; see e.g. [Pruthi et al., 2019] for a recent survey.”
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization (2021, background): “As a result, models are brittle to rare words (Gong et al., 2018) and perturbations, both natural and adversarial (Belinkov and Bisk, 2018; Pruthi et al., 2019; Sun et al., 2020).”
Contextualized Perturbation for Textual Adversarial Attack (2020, background): “…(Liang et al., 2019; Ebrahimi et al., 2018; Li et al., 2018; Gao et al., 2018, inter alia ) use misspellings to attack the victim systems; however, these attacks can often be defended by a spell checker (Pruthi et al., 2019; Vijayaraghavan and Roy, 2019; Zhou et al., 2019b; Jones et al., 2020).”
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP (2020, methodology): “…new methods for generating adversarial examples and better approaches to defending against these adversaries (Alzantot et al., 2018; Jin et al., 2019; Kuleshov et al., 2018; Li et al., 2019; Gao et al., 2018; Wang et al., 2019; Ebrahimi et al., 2017; Zang et al., 2020; Pruthi et al., 2019).”
Evaluating the Robustness of Neural Language Models to Input Perturbations (2021, background): “Moreover, most of previous work on adversarial perturbation to NLP models focused on misspelling attacks (Jones et al., 2020; Pruthi et al., 2019; L. Sun et al., 2020).”
The Fallacy of AI Functionality (2022, background): “For example, when large language models are usedto processthe queries of AI-powered web search [131], the models’fragility tomisspellings [125, 147], ortrivial changes to format [19] and context [18] can lead to unexpected results.”
On Robust Prefix-Tuning for Text Classification (2022, background): “…Jones et al., 2020), certiﬁcation (Jia et al., 2019; Huang et al., 2019; Shi et al., 2020; Xu et al., 2020; Ye et al., 2020), adversary detection (Pruthi et al., 2019; Zhou et al., 2019), and adversarial training (Miyato et al., 2017; 2019; Zhu et al., 2020; Jiang et al., 2020a; Liu et al.,…”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, neutral): “The threat models employed in this literature include synonym-based attacks [62–64], character-based substitutions [24], and spelling mistakes [65].”
Certified Robustness to Adversarial Word Substitutions (2019, background): “In principle, IBP can handle character-level typos (Ebrahimi et al., 2017; Pruthi et al., 2019), though typos yield more perturbations per word than we consider in this work.”
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing (2021, background): “Transformer-based PTLMs are brittle to noise which includes both adversarial and natural noise [291], [292] .”
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning (2020, background): “For character-level attacks, speciﬁcally-designed model modules (Pruthi et al., 2019; Jones et al., 2020) and pre-training tasks (Ma et al., 2020) have been proposed and have been shown to work well in handling adversarial typos.”
A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement (2022, background): “Different from that in the image ﬁeld, attacks that result in illegal text sentences can be easily detected by spelling and grammar error correction [ Pruthi et al. , 2019 ] .”
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters (2020, background): “This improved robustness is desirable since BERT seems to be sensitive to misspellings (Pruthi et al., 2019; Sun et al., 2020).”
BERT-ATTACK: Adversarial Attack against BERT Using BERT (2020, background): “Works such as (Wallace et al., 2019; Jin et al., 2019; Pruthi et al., 2019) have explored these pre-trained language models in many different angles.”