"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset

19 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, background): “Similarly, HolisticBias dataset (Smith et al., 2022) is a collection of author-crafted American-English prompts which contains 600 descriptor terms across 13 different demographics.”
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “…in particular, to generate harmful or inappropriate content (Bender et al., 2021; Bommasani et al., 2021; Hendrycks et al., 2021; Weidinger et al., 2021), including work from our group (Xu et al., 2020; Dinan et al., 2022, 2021; Smith et al., 2022; Dinan et al., 2020a; Smith and Williams, 2021).”
Toxicity in Multilingual Machine Translation at Scale (2022, methodology): “For this purpose, we combine the previously defined toxicity detection methodology (NLLB Team et al., 2022), the controlled evaluation dataset H OLISTIC B IAS (Smith et al., 2022), and the ALTI + interpretability method (Ferrando et al., 2022a).”