Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community

By Ethan Fast, Tina Vachovsky, Michael S. Bernstein. 2016

75 citations recorded by the source corpus.

Imagine a princess asleep in a castle, waiting for her prince to slay the dragon and rescue her. Tales like the famous Sleeping Beauty clearly divide up gender roles. But what about more modern stories, borne of a generation increasingly aware of social constructs like sexism and racism? Do these stories tend to reinforce gender stereotypes, or counter them? In this paper, we present a technique that combines natural language processing with a crowdsourced lexicon of stereotypes to capture gender biases in fiction. We apply this technique across 1.8 billion words of fiction from the Wattpad online writing community, investigating gender representation in stories, how male and female characters behave and are described, and how authors' use of gender stereotypes is associated with the community's ratings. We find that male over-representation and traditional gender stereotypes (e.g., dominant men and submissive women) are common throughout nearly every genre in our corpus. However, only some of these stereotypes, like sexual or violent men, are associated with highly rated stories. Finally, despite women often being the target of negative stereotypes, female authors are equally likely to write such stereotypes as men.

Subfields: Artificial Intelligence, Gender Studies, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Gender Studies

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “, 2019) and its open-source replica OWTC (Gokaslan and Cohen, 2019), inspired by previous work in analyzing social biases in large text corpora (Fast et al., 2016).”
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, neutral): “Current language models can perpetrate harms such as the cyclical erasure of non-binary gender identities (Uppunda et al., 2021; Sap, 2021; Lakoff; Fiske, 1993; Fast et al., 2016; Behm-Morawitz and Mastro, 2008).”
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023, background): “This is especially problematic in light of the potential real-world harms that ensue [8, 9, 13, 31].”
Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts (2019, background): “…of Bolukbasi et al. (2016), grouped across definitional and stereotypical gender (male, female, neutral), (4) the gender stereotype lexicon of Fast et al. (2016), (5) the gender lexicon for social media from (Sap et al., 2014), (6) a manually-compiled corpus of gendered words, extended from…”
Connotation Frames of Power and Agency in Modern Films (2017, background): “Fast et al. (2016) examine the stereotypes present in fan-ﬁction us-7 ing a lexicon-based strategy that focus on commonly gender-biased attributes (e.g., emotional for women) rather than the overall power dynamics of the story.”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, background): “Our findings on movie scripts show the promise of using Controllable Debiasing to successfully mitigate gender biases in portrayal of characters, which could be extended to other domains (e.g., news or fiction, Field and Tsvetkov, 2019; Fast et al., 2016).”
The Birth of Bias: A case study on the evolution of gender bias in an English language model (2022, unknown): “Following others (Zhao et al., 2019; Tan and Celis, 2019; Fast et al., 2016; Gao et al., 2020), we examine the word-count statistics for the dataset in our experiments.”
Gender and Representation Bias in GPT-3 Generated Stories (2021, methodology): “For measuring power, we take Fast et al. (2016b)’s lexicons for strong and dominant (113 words), and contrast them with a union of their lexicons for weak, dependent, submissive, and afraid (141 words).”
Unsupervised Discovery of Implicit Gender Bias (2020, methodology): “Comparison to stereotype lexicons In order to better understand the trends in the influential word list, we draw from prior work on stereotype detection and align our model’s predictions with existing stereotypes (Fast et al., 2016).”
Automatically Inferring Gender Associations from Language (2019, background): “…people talk to women and men (Voigt et al., 2018), as well as how people talk about women and men, typically in specific domains such as sports journalism (Fu et al., 2016), fiction writing (Fast et al., 2016), movie scripts (Sap et al., 2017), and Wikipedia biographies (Wagner et al., 2015, 2016).”
Attesting Biases and Discrimination using Language Semantics (2019, background): “[8] found that modern amateur writers (male and female) are still reinforcing traditional gender stereotypes in their work, by picturing female characters as submissive and male characters as dominant.”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021, background): “However, in both application areas, the training data has been shown to have problematic characteristics [18, 38, 42, 47, 61] resulting in models that encode stereotypical and derogatory associations along gender, race, ethnicity, and disability status [11, 12, 69, 69, 132, 132, 157].”
Entity-Centric Contextual Affective Analysis (2019, background): “…people portray-als take either an unsupervised approach (Bam-man et al., 2013), which requires large amounts of data and can be difﬁcult to interpret, or rely on domain-speciﬁc knowledge (Fast et al., 2016; Wagner et al., 2015), which does not generalize well to other hypotheses and data domains.”
Judging a Book by its Description : Analyzing Gender Stereotypes in the Man Bookers Prize Winning Fiction (2018, background): “(Fast et al., 2016) have worked on gender stereotypes in English fiction particularly on the Online Fiction Writing Community.”
RtGender: A Corpus for Studying Differential Responses to Gender (2018, background): “For instance, women 1 journal-ists reach a smaller audience in terms of social media impressions (Matias and Wallach, 2012), and traditional gender stereotypes and unbalanced gender representation occur even in contemporary stories and movies (Fast et al., 2016; Sap et al., 2017).”
A Survey on Gender Bias in Natural Language Processing (2021, methodology): “Otherwise, modelling gender as a binary variable can lead to a number of harms such as misgendering and erasure via invalidation or obscuring of non-binary gender identities [Behm-Morawitz and Mastro 2008; Fast et al. 2016].”