Darling or Babygirl ? Investigating Stylistic Bias in Sentiment Analysis

By Judy Hanwen Shen, Lauren Fratamico, Iyad Rahwan, Alexander M. Rush. 2018

Subfields: Sentiment & Opinion Analysis, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Sentiment & Opinion Analysis

1. Eduard H. Hovy
2. Anthony Rios
3. Zachary Chase Lipton
4. Divyansh Kaushik

In AI Ethics, Fairness & Society

1. Eduard H. Hovy
2. William Yang Wang
3. Douwe Kiela
4. Anthony Rios
5. Zachary Chase Lipton

Citations in the corpus

• Learning the Difference that Makes a Difference with Counterfactually-Augmented Data (2019, background): “[26] substitute words to test the behavior of sentiment analysis algorithms in the presence of stylistic variation, finding that similar word pairs produce significant differences in sentiment score.”
• Towards Controllable Biases in Language Generation (2020, neutral): “, 2017), sentiment analysis (Shen et al., 2018), and more.”
• Dats Wassup!!: Investigating African-American Vernacular English in Transformer-Based Text Generation (2020, methodology): “Sheng et al. (2019) shows that NLG tools are biased with regard to the subject of a sentence when that subject belongs to an underprivileged group, and Shen et al. (2018) tests sentiment analysis tools with intent-controlled pairs with varying stylistic inclinations.”
• Investigating Societal Biases in a Poetry Composition System (2020, background): “…et al., 2018; Zhao et al., 2018a), semantic role labeling (Zhao et al., 2017), abusive language detection (Park et al., 2018), sentiment analysis (Shen et al., 2018; Kiritchenko and Mohammad, 2018), and language modeling (Sheng et al., 2019; Bordia and Bowman, 2019; Pryzant et al., 2020; Huang…”
• DynaSent: A Dynamic Benchmark for Sentiment Analysis (2020, background): “It has, however, entered into a more challenging phase for research and technology development: while present-day models achieve outstanding re-sults on all available benchmark tasks, they still fall short when deployed as part of real-world systems (Burn-Murdoch, 2013; Grimes, 2014, 2017; Gossett, ”
• FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English (2020, background): “Moreover, in this work we focus on fairness ranking whereas Shen et al. (2018) studied the impact of model changes across various groups of words.”
• Theories of “Gender” in NLP Bias Research (2022, background): “05541v1 [133] Judy Hanwen Shen, Lauren Fratamico, Iyad Rahwan, and Alexander M Rush.”
• Explaining The Efficacy of Counterfactually-Augmented Data (2020, background): “…shown to be vulnerable to synthetic transformations, such as distractor phrases (Jia & Liang, 2017; Wallace et al., 2019), document paraphrases (Iyyer et al., 2018; Pfeiffer et al., 2019), and meaning preserving synthetic modifications (Ribeiro et al., 2018; Glockner et al., 2018; Shen et al., 2018).”