A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement

By Yuting Yang, Pei Huang, Juan Cao, Jintao Li, Yun Lin, Jin Song Dong, Feifei Ma, Jian Zhang. 2022

18 citations recorded by the source corpus.

Recent years have seen the wide application of NLP models in crucial areas such as finance, medical treatment, and news media, raising concerns of the model robustness and vulnerabilities. In this paper, we propose a novel prompt-based adversarial attack to compromise NLP models and robustness enhancement technique. We first construct malicious prompts for each instance and generate adversarial examples via mask-and-filling under the effect of a malicious purpose. Our attack technique targets the inherent vulnerabilities of NLP models, allowing us to generate samples even without interacting with the victim NLP model, as long as it is based on pre-trained language models (PLMs). Furthermore, we design a prompt-based adversarial training method to improve the robustness of PLMs. As our training method does not actually generate adversarial samples, it can be applied to large-scale training sets efficiently. The experimental results show that our attack method can achieve a high attack success rate with more diverse, fluent and natural adversarial examples. In addition, our robustness enhancement method can significantly improve the robustness of models to resist adversarial attacks. Our work indicates that prompting paradigm has great potential in probing some fundamental flaws of PLMs and fine-tuning them for downstream tasks.

Subfields: Artificial Intelligence, Prompting & Reasoning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Derek Tam
3. Colin Raffel
4. Haokun Liu
5. Luke S. Zettlemoyer
6. Mohammed Muqeeth
7. Tenghao Huang
8. Jay Mohta

In Prompting & Reasoning

1. Derek Tam
2. Mohit Bansal
3. Colin Raffel

In Safety, Robustness & Interpretability

1. Mohit Bansal
2. Colin Raffel
3. Haokun Liu
4. Greg Durrett
5. Xi Ye
6. Yichen Jiang

Citations in the corpus

• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “Prompt tuning has been applied to various applications for NLP including continual learning [59], model robustness [60, 61], summarization [62], machine translation [63], co-training [64], probing language models [65, 65], inverse prompting [66] and transfer learning [67].”