Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting
By Zahra Fatemi, Chen Xing, Wenhao Liu, Caiming Xiong. 2021
45 citations recorded by the source corpus.
Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such data. However, given the limited size and concentrated focus of the gender-neutral data, catastrophic forgetting would occur during second-phase pre-training. Forgetting information in the original training data may damage the model’s downstream performance by a large margin. In this work, we empirically show that catastrophic forgetting occurs in such methods by evaluating them with general NLP tasks in GLUE. Then, we propose a new method, GEnder Equality Prompt (GEEP), to improve gender fairness of pre-trained models with less forgetting. GEEP freezes the pre-trained model and learns gender-related prompts with gender-neutral data.Empirical results show that GEEP not only achieves SOTA performances on gender fairness tasks, but also forgets less and performs better on GLUE by a large margin.
Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohammad Shoeybi
- Bryan Catanzaro
- Yuxiong He
- Samyam Rajbhandari
- M. Patwary
- Anima Anandkumar
- Jared Casper
- Vijay Anand Korthikanti
In Natural Language Processing
- Mohammad Shoeybi
- M. Patwary
- Tong Yu
- Ryan A. Rossi
- Ruiyi Zhang
- Anima Anandkumar
- Sungchul Kim
- Shaden Smith
In Safety, Robustness & Interpretability
- Mohammad Shoeybi
- Anima Anandkumar
- Bryan Catanzaro
- Franck Dernoncourt
- Rafal Kocielnik
- M. Patwary
- Shrimai Prabhumoye
Citations in the corpus
- Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, background): “c) Prompt engineering – where the inputs to the model for each query are modified to steer the model away from bias [60, 14].”
- Bias and Fairness in Large Language Models: A Survey (2023, methodology): “As a bias mitigation technique, Fatemi et al. (2023) propose GEEP to use continuous prompt tuning to mitigate gender bias, fine-tuning on a gender-neutral dataset. In Yang et al. (2023)’s ADEPT technique, continuous prompts encourage neutral nouns and adjectives to be independent of protected attrib”
- Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases (2021, background): “Finetuning a model can also lead to over-fitting especially in case of smaller train sets and to catastrophic forgetting of knowledge present in the pre-trained model (Fatemi et al., 2021).”