Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models

By Tong Niu, Mohit Bansal. 2018

92 citations recorded by the source corpus.

We present two categories of model-agnostic adversarial strategies that reveal the weaknesses of several generative, task-oriented dialogue models: Should-Not-Change strategies that evaluate over-sensitivity to small and semantics-preserving edits, as well as Should-Change strategies that test if a model is over-stable against subtle yet semantics-changing modifications. We next perform adversarial training with each strategy, employing a max-margin approach for negative generative examples. This not only makes the target dialogue model more robust to the adversarial inputs, but also helps it perform significantly better on the original inputs. Moreover, training on all strategies combined achieves further improvements, achieving a new state-of-the-art performance on the original task (also verified via human evaluation). In addition to adversarial training, we also address the robustness task at the model-level, by feeding it subword units as both inputs and outputs, and show that the resulting model is equally competitive, requires only 1/4 of the original vocabulary size, and is robust to one of the adversarial strategies (to which the original model is vulnerable) even without adversarial training.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “This over-sensitivity to minimal changes has also been observed by Niu & Bansal (2018).”
Automatically Learning Data Augmentation Policies for Dialogue Tasks (2019, methodology): “Moreover, all best policies include Random Swap, which agrees with the results in Niu and Bansal (2018).”
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models (2020, background): “sequence-to-sequence based dialog system models, such as their over-sensitivity and over-stability (Niu and Bansal, 2018), robustness (Cheng et al.”
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020, background): “Dialogue systems have been shown to be sensitive to the input messages (Niu and Bansal, 2018; Zhang et al., 2020; Xu et al., 2020).”
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, background): “…2020), inserting random tokens (Wei and Zou, 2019; Miao et al., 2020), replacing non-important tokens with random tokens (Xie et al., 2017, 2020; Niu and Bansal, 2018) or randomly swapping to-kens in one sentence (Artetxe et al., 2018; Lample et al., 2018; Wei and Zou, 2019; Miao et al., 2020)…”
Robust Conversational Agents against Imperceptible Toxicity Triggers (2022, background): “Cheng et al. (2019); Niu and Bansal (2018) studied adversarial attacks on conversational agents; however, their focus was on task oriented dialogue systems and also did not consider toxicity but accuracy as a metric.”
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (2023, methodology): “Prior work employs various heuristics, such as negation injection (Niu & Bansal, 2018; Kassner et al., 2021; Gubelmann & Handschuh, 2022) and entity substitution (Longpre et al., 2021; Zhou et al., 2023), and finds that language models (both large and small) tend to be stubborn and cling to their…”
Data Boost: Text Data Augmentation through Reinforcement Learning Guided Conditional Generation (2020, unknown): “(Niu and Bansal, 2018) (synonyms + antonym from WordNet) 0.”
Effective Data Augmentation Approaches to End-to-End Task-Oriented Dialogue (2019, background): “The variance and noise in user utterances can prevent the system from over-sensitivity [18], thus making the system more robust.”
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020, background): “…set, we also add these samples back to the training set, similar to the concept of adversarial training (Jia and Liang, 2017; Wang et al., 2019c; Niu and Bansal, 2018) where the adversarial examples are added back to the training set so that the resulting model will be more robust to similar…”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, background): “Furthermore, [99] attacked the neural models for dialogue generation.”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “The work in (Niu & Bansal, 2018) explores the oversensitivity and over-stability of neural dialogue models by using some heuristic techniques to modify original inputs and observe the corresponding outputs.”