Social Bias in Elicited Natural Language Inferences

By Rachel Rudinger, Chandler May, Benjamin Van Durme. 2017

140 citations recorded by the source corpus.

We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The SNLI human-elicitation protocol makes it prone to amplifying bias and stereotypical associations, which we demonstrate statistically (using pointwise mutual information) and with qualitative examples.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, methodology): “Our work has focused on producing a dataset with minimal annotation artifacts, which in turn helps to avoid some gender and racial biases that stem from elicitation (Rudinger et al., 2017).”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, background): “See Table 5 in Appendix A for a complete tag taxonomy. of them contain text or annotations that were collected in uncontrolled settings, they contain evidence of stereotypes and biases that one may not wish one’s system to learn (Rudinger et al., 2017).”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “Stereotyping Bolukbasi et al. (2016a,b); Caliskan et al. (2017); McCurdy and Serbetçi (2017); Rudinger et al. (2017); Zhao et al. (2017); Curry and Rieser (2018); Díaz et al. (2018); Santana et al. (2018); Sutton et al. (2018); Zhao et al. (2018a,b); Agarwal et al. (2019); Basta et al. (2019);…”
Gender Bias in Coreference Resolution (2018, background): “Prior work also analyzes social and gender stereotyping in existing NLP and vision datasets (van Miltenburg, 2016; Rudinger et al., 2017).”
Annotation Artifacts in Natural Language Inference Data (2018, background): “Rudinger et al. (2017) revealed that elicited hypotheses in SNLI contain evidence of various gender, racial, religious, and aged-based stereotypes.”
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts (2021, background): “, 2020), as well as other social biases (Caliskan et al., 2017; Rudinger et al., 2017; Sap et al., 2020; Sotnikova et al., 2021).”
Toward Gender-Inclusive Coreference Resolution (2019, background): “Gender bias in NLP has been considered more broadly than just in coreference resolution, including, for instance, natural language inference [Rudinger et al. 2017], word embeddings [e.”
On Measuring and Mitigating Biased Inferences of Word Embeddings (2019, background): “(Rudinger, May, and Van Durme 2017), (Gururangan et al. 2018) and references therein point out that the mechanism for gathering the SNLI data allows various stereotypes (gender, age, race, etc.) and annotation artifacts to seep into the data.”
Evaluating Gender Bias in Machine Translation (2019, background): “…done by women, construction workers are stereotypically men; Zhao et al., 2017), lexical semantics (“man is to computer programmer as woman is to home-maker”; Bolukbasi et al., 2016), and natural language inference (associating women with gossiping and men with guitars; Rudinger et al., 2017).”
Emergent Abilities of Large Language Models (2022, neutral): “On WinoGender (Rudinger et al., 2017), which measures gender bias in occupations such as “nurse” or “electrician,” scaling has improved performance so far (Du et al., 2021; Chowdhery et al., 2022), though BIG-Bench (2022) found in BBQ bias benchmark (Parrish et al., 2022) that bias can increase…”
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, background): “learn and use these biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2017; May et al., 2010; Zhao et al., 2018; Rudinger et al., 2017).”
Hypothesis Only Baselines in Natural Language Inference (2018, background): “1 Concerns have been raised previously about the hypotheses in the Stanford Natural Language Inference (SNLI) dataset speciﬁcally, such as by Rudinger et al. (2017) and in unpublished work.”
On Measuring Social Biases in Sentence Encoders (2019, background): “…at the intersection of race and gender in distributional semantic word representations (Her-belot et al., 2012), natural language inference data (Rudinger et al., 2017), and facial recognition systems (Buolamwini and Gebru, 2018), as well as at the intersection of dialect and gender in…”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “Neural networks learn to replicate historical, societal biases from training data in various tasks such as natural language inference (Rudinger et al., 2017), coreference resolution (Cao and Daum´e III, 2019), and sentiment analysis (Kiritchenko and Mohammad, 2018).”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, background): “21This line of work is complementary to other notions of dataset bias, like understanding what phenomena datasets cover or don’t [76], particularly how that relates to how marginalized groups are represented and portrayed [71, 90, 69, 68].”
Learning Gender-Neutral Word Embeddings (2018, background): “Rudinger et al. (2017) use point-wise mutual information to test the SNLI (Bow-man et al., 2015) corpus and demonstrate gender stereotypes as well as varying degrees of racial, re-ligious, and age-based stereotypes in the corpus.”
Assessing Social and Intersectional Biases in Contextualized Word Representations (2019, background): “Social bias in language has been demonstrated to exist upstream in a variety of corpora and datasets; on a corpus elicited from crowdworkers [28], and on the 1 Billion Word Benchmark corpus [7] where it was observed that there was gender skew in proportions of gendered pronouns and associations with”
Multi-Dimensional Gender Bias Classification (2020, background): “In particular, NLP models often learn biases against others based on their gender (Bolukbasi et al., 2016; Hovy and Spruit, 2016; Caliskan et al., 2017; Rudinger et al., 2017; Garg et al., 2018; Gonen and Goldberg, 2019; Dinan et al., 2019a).”