Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis

By Jayadev Bhaskaran, Isha Bhallamudi. 2019

52 citations recorded by the source corpus.

In this work, we investigate the presence of occupational gender stereotypes in sentiment analysis models. Such a task has implications in reducing implicit biases in these models, which are being applied to an increasingly wide variety of downstream tasks. We release a new gender-balanced dataset of 800 sentences pertaining to specific professions and propose a methodology for using it as a test bench to evaluate sentiment analysis models. We evaluate the presence of occupational gender stereotypes in 3 different models using our approach, and explore their relationship with societal perceptions of occupations.

Subfields: Artificial Intelligence, General Social Sciences, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…et al. (2018); Santana et al. (2018); Sutton et al. (2018); Zhang et al. (2018); Zhao et al. (2018a,b); Agarwal et al. (2019); Basta et al. (2019); Bhaskaran and Bhallamudi (2019); Brunet et al. (2019); Cao and Daumé (2019); Chaloner and Maldonado (2019); Dev and Phillips (2019); Ethayarajh et…”
Release Strategies and the Social Impacts of Language Models (2019, background): “Biases are reinforced from a myriad of directions; occupational gender stereotypes are an example of social bias well ingrained by external influences like mass media [6].”
A Survey on Gender Bias in Natural Language Processing (2021, methodology): “For datasets where each instance contains at least two versions of the same template sentence, e.g., male and female, the paired t-test has been used to measure if the mean predicted class probabilities are different across genders [Bhaskaran and Bhallamudi 2019; Kiritchenko and Mohammad 2018].”
Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation (2020, background): “This unwanted bias can cause biased results in the downstream tasks (Caliskan et al., 2017a; Kiritchenko and Mohammad, 2018; Bhaskaran and Bhallamudi, 2019) and gender discrimination in NLP systems.”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, unknown): “, 2017), 11 (Blodgett and O’Connor, 2017; Bhaskaran and Bhallamudi, 2019), 12 (De-Arteaga et al.”