Social Bias Frames: Reasoning about Social and Power Implications of Language

By Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, Yejin Choi. 2019

640 citations recorded by the source corpus.

Warning: this paper contains content that may be offensive or upsetting. Language has the power to reinforce stereotypes and project social biases onto others. At the core of the challenge is that it is rarely what is stated explicitly, but rather the implied meanings, that frame people’s judgments about others. For example, given a statement that “we shouldn’t lower our standards to hire more women,” most listeners will infer the implicature intended by the speaker - that “women (candidates) are less qualified.” Most semantic formalisms, to date, do not capture such pragmatic implications in which people express social biases and power differentials in language. We introduce Social Bias Frames, a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and stereotypes onto others. In addition, we introduce the Social Bias Inference Corpus to support large-scale modelling and evaluation with 150k structured annotations of social media posts, covering over 34k implications about a thousand demographic groups. We then establish baseline approaches that learn to recover Social Bias Frames from unstructured text. We find that while state-of-the-art neural models are effective at high-level categorization of whether a given statement projects unwanted social bias (80% F1), they are not effective at spelling out more detailed explanations in terms of Social Bias Frames. Our study motivates future work that combines structured pragmatic inference with commonsense reasoning on social implications.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…(2019); Sweeney and Najafian (2019); Vaidya et al. (2019); Gaut et al. (2020); Gencoglu (2020); Hovy et al. (2020); Hutchinson et al. (2020); Kim et al. (2020); Peng et al. (2020); Rios (2020); Sap et al. (2020); Shah et al. (2020); Sheng et al. (2020); Tan et al. (2020); Zhang et al. (2020a,b).”
LaMDA: Language Models for Dialog Applications (2022, background): “, [58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72]), and inappropriately revealing personally identifying information (PII) from training data [73].”
Challenges in Detoxifying Language Models (2021, methodology): “A second type of bias considers disparate performance across dialects, where classifiers on average assign higher toxicity scores e.g. to African-American English (AAE) (Davidson et al., 2019; Sap et al., 2019a).”
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, background): “Additionally, measures of bias could be multi-dimensional (e.g., Dinan et al., 2020), include explanations (e.g., Sap et al., 2020), or be evolving over time (e.g., using similarity to toxic online content).”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “I16: “hate has always been expressed idiomatically”) and closely resembles “implied bias” (Sap et al., 2020) as well”
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “The manual collection relies on human annotators to compose new samples or label existing data (Forbes et al., 2020; Hendrycks et al., 2020; Sap et al., 2020; Lourie et al., 2021; Emelin et al., 2021).”
The Capacity for Moral Self-Correction in Large Language Models (2023, background): “Large language models exhibit harmful social biases [1, 6, 8, 11, 15, 24, 29, 50, 62] that can sometimes get worse for larger models [2, 18, 20, 43, 55].”
Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech (2023, methodology): “Recent work has been proposed to apply the Generative Pre-trained models1, to create NLEs describing why a given text is considered to be biased [16] or hateful [8, 16].”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Large language models exhibit a wide range of harmful behaviors such as reinforcing social biases (e.g., [47, 28, 1, 33, 7]), generating offensive or toxic outputs [25], leaking personally identiﬁable information from the training data [13], aiding in disinformation campaigns [12], generating…”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “%) on 3 existing human -written implicit toxic datasets: Im-plicitHateCorpus (ElSherief et al., 2021), SocialBi-asFrames (Sap et al., 2020), and DynaHate (Vidgen et al., 2021).”
BBQ: A hand-built bias benchmark for question answering (2021, background): “LMs have been found to reproduce social biases in downstream tasks, such as textual entailment (Dev et al., 2020; Sharma et al., 2021), language generation (Sheng et al., 2019), and coreference resolution (Rudinger et al., 2018).”
A Multimodal Framework for the Detection of Hateful Memes (2020, methodology): “Similarly to SBIC, we trained on batches of the primary and auxiliary tasks sampled in different ratios, and with different classiﬁcation heads for each task.”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “, 2021), social bias implications (Sap et al., 2020), and toxic speech (Gehman et al.”
Multi-Dimensional Gender Bias Classification (2020, background): “Sap et al. (2020) proposed a different but compatible perspective, and aimed to situate gender bias within the broader landscape of negative stereotypes in social media text, an approach that can make parallels apparent across different kinds of harmful text.”
PaLM 2 Technical Report (2023, background): “Evaluations of toxicity classification capabilities may not generalize to other constructs important for supporting responsible AI practices (Sap et al., 2020; Thoppilan et al., 2022).”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, background): “Therefore, we hope future work continues to better deﬁne and evaluate the presence of harmful language (e.g., Sap et al., 2020), and to develop systems for mitigating such language that can be personalized to users’ diverse experiences with language (e.g., dealing with reclaimed slurs…”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “Alternatively, we recommend exploring more holistic representations of social biases or toxicity (e.g., Social Bias Frames; Sap et al., 2020).”