Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases

By Shrimai Prabhumoye, Rafal Kocielnik, Mohammad Shoeybi, Anima Anandkumar, Bryan Catanzaro. 2021

27 citations recorded by the source corpus.

Detecting social bias in text is challenging due to nuance, subjectivity, and difficulty in obtaining good quality labeled datasets at scale, especially given the evolving nature of social biases and society. To address these challenges, we propose a few-shot instruction-based method for prompting pre-trained language models (LMs). We select a few class-balanced exemplars from a small support repository that are closest to the query to be labeled in the embedding space. We then provide the LM with instruction that consists of this subset of labeled exemplars, the query text to be classified, a definition of bias, and prompt it to make a decision. We demonstrate that large LMs used in a few-shot context can detect different types of fine-grained biases with similar and sometimes superior accuracy to fine-tuned models. We observe that the largest 530B parameter model is significantly more effective in detecting social bias compared to smaller models (achieving at least 13% improvement in AUC metric compared to other models). It also maintains a high AUC (dropping less than 2%) when the labeled repository is reduced to as few as $100$ samples. Large pretrained language models thus make it easier and quicker to build new bias detectors.

Subfields: Artificial Intelligence, General Social Sciences, Prompting & Reasoning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Prompting & Reasoning

In Safety, Robustness & Interpretability

Citations in the corpus

Predictability and Surprise in Large Generative Models (2022, background): “g, How do you ensure the system does not generate inappropriate and harmful outputs, such as making overtly sexist or racist comments [65]? How do you identify bias issues in the system prior to deployment [8, 53]? How do you ensure that when the model outputs a claim, it isn’t making up facts [10]?”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “…2016; Caliskan et al., 2017; Nadeem et al., 2021) and expressed in specific tasks (Nangia et al., 2020; Kirk et al., 2021; Schramowski et al., 2022; Prabhumoye et al., 2021; Srinivasan and Bisk, 2021; Kirk et al., 2021; Parrish et al., 2021; Baldini et al., 2022; Czarnowska et al., 2021; Dev et…”
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models (2022, background): “Such biases may be extracted by certain forms of prompting; recent work (Prabhumoye et al., 2021) suggests that fewshot prompts can be used to detect social biases in pretrained language models.”