HONEST: Measuring Hurtful Sentence Completion in Language Models

By Debora Nozza, Federico Bianchi, Dirk Hovy. 2021

194 citations recorded by the source corpus.

Language models have revolutionized the field of NLP. However, language models capture and proliferate hurtful stereotypes, especially in text generation. Our results show that 4.3% of the time, language models complete a sentence with a hurtful word. These cases are not random, but follow language and gender-specific patterns. We propose a score to measure hurtful sentence completions in language models (HONEST). It uses a systematic template- and lexicon-based bias evaluation methodology for six languages. Our findings suggest that these models replicate and amplify deep-seated societal stereotypes about gender roles. Sentence completions refer to sexual promiscuity when the target is female in 9% of the time, and in 4% to homosexuality when the target is male. The results raise questions about the use of these models in production settings.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “languages (Nozza et al., 2021), and discriminatory gender biases were found in contextual word embedding by BERT (Kurita et al., 2019) and ELMo (Zhao et al., 2019).”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “1 Hate, Toxicity, and Targeted Violence Beyond hate speech and toxic language, generations may also produce harmful biases [87], stereotypes [165] (overlapping with 4.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, positive): “Evidence for this behavior has been substantially corroborated in NLP literature (Nadeem et al., 2021; Nozza et al., 2021; Huang et al., 2020b; Hassan et al., 2021; Blodgett et al., 2020; Liang et al., 2021; Field et al., 2021, inter alia).”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, background): “In particular, many works focus on generative models (Dinan et al., 2020a,b; Xu et al., 2021b; Kirk et al., 2021; Sheng et al., 2021b; Nozza et al., 2021; Renduchintala et al., 2021; Baheti et al., 2021; Perez et al., 2022), which are well known to pose unique challenges for automatic evaluation…”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “This approach is similar to the HONEST score by Nozza et al. (2021).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “However, these classifiers themselves can have issues with bias, e.g., by learning undesirable correlations that tie toxicity to identity terms (Dixon et al., 2018; Nozza et al., 2021, 2022), or language varieties, such as African American English (Liu et al., 2019; Sap et al., 2019).”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, methodology): “This approach is similar to the HONEST score by Nozza et al. (2021).”
“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation (2023, background): “[42] Debora Nozza, Federico Bianchi, and Dirk Hovy.”
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models (2023, methodology): “Nozza et al. (2021, 2022) use template-based tests for evaluating bias in text completions.”
A Survey on Gender Bias in Natural Language Processing (2021, background): “Findings made for English do not automatically extend to other languages, especially if those exhibit morphological gender agreement [Nozza et al. 2021].”
Large Language Model Alignment: A Survey (2023, background): “, 2020), studies on a series of benchmarks (Nadeem et al., 2020; Nangia et al., 2020; Nozza et al., 2021) confirm that LLMs exhibit varying degrees of stereotypes related to gender, social bias, culture, and race.”
Auditing large language models: a three-layered approach (2023, background): “For example, several studies have documented situations in which LLMs propagate toxic language [162], [236], but the interpretation of toxicity and the materialisation of its harms vary across cultural, social, or political groups [237]–[239].”
Five sources of bias in natural language processing (2021, background): “Additional metrics, approaches and data sets have been proposed to measure the bias inherent in large language models and their sentence completions (Nangia et al., 2020; Nozza et al., 2021; Sheng et al., 2019).”
Probing Toxic Content in Large Pre-Trained Language Models (2021, background): “Extensions of our work should take this shortcoming into account and consider probing content with regard to more communities, religions and ideologies, as well as non-binary people as previously expressed by Mohammad (2020) and Nozza et al. (2021).”
Pipelines for Social Bias Testing of Large Language Models (2022, background): “Then, it is possible to exploit lexicons (Nozza et al., 2021, 2022), or hate speech (Ousidhoum et al., 2021; Sheng et al., 2019) and sentiment classifiers Hutchinson et al. (2020); Huang et al. (2020) to obtain a social bias score from the template-based generated text.”
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals (2022, methodology): “We also release an extension of the benchmark framework HONEST (Nozza et al., 2021) with a novel set of identity terms related to LGBTQIA+ community.”
On the Gap between Adoption and Understanding in NLP (2021, background): “However, papers soon highlighted its limits (Ettinger, 2020; Bender and Koller, 2020) and identiﬁed issues with bias (Nozza et al., 2021), i.e., that contextual models show different performances for different genders and ethnic groups (Zhang et al., 2020).”
"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset (2022, background): “In particular, many works have focused squarely on generative models (Perez et al., 2022; Xu et al., 2021b; Kirk et al., 2021a; Sheng et al., 2021b; Nozza et al., 2021; Renduchintala et al., 2021; Dinan et al., 2020a,b), which are well known to pose unique challenges for automatic evaluation (Lowe…”