Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts

By Luke Breitfeller, Emily Ahn, David Jurgens, Yulia Tsvetkov. 2019

143 citations recorded by the source corpus.

Microaggressions are subtle, often veiled, manifestations of human biases. These uncivil interactions can have a powerful negative impact on people by marginalizing minorities and disadvantaged groups. The linguistic subtlety of microaggressions in communication has made it difficult for researchers to analyze their exact nature, and to quantify and extract microaggressions automatically. Specifically, the lack of a corpus of real-world microaggressions and objective criteria for annotating them have prevented researchers from addressing these problems at scale. In this paper, we devise a general but nuanced, computationally operationalizable typology of microaggressions based on a small subset of data that we have. We then create two datasets: one with examples of diverse types of microaggressions recollected by their targets, and another with gender-based microaggressions in public conversations on social media. We introduce a new, more objective, criterion for annotation and an active-learning based procedure that increases the likelihood of surfacing posts containing microaggressions. Finally, we analyze the trends that emerge from these new datasets.

Subfields: Artificial Intelligence, Communication, Sociology and Political Science, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Sociology and Political Science

In Natural Language Processing

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, background): “, 2017), or microaggressions (Breitfeller et al., 2019).”
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, background): “Our analyses confirm that determining whether a document has toxic or lewd content is a more nuanced endeavor that goes beyond detecting “bad” words; hateful and lewd content can be expressed without negative keywords (e.g., microaggressions, innuendos; Breitfeller et al., 2019; Dinan et al., 2019).”
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, background): “These harmful representations are usually overlooked by toxic language detectors (Sap et al., 2019; Breitfeller et al., 2019; Hartvigsen et al., 2022), while they can resurface in language technologies and disadvantage an already disadvantaged group of people.”
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022, methodology): “To encourage implicit toxicity from a LLM, we ﬁnd examples of human-written sentences with implicit toxicity towards each group from hate forums (de Gibert et al., 2018) and Reddit (Breitfeller et al., 2019).”
Improving alignment of dialogue agents via targeted human judgements (2022, background): “We base our microaggresion rule on Breitfeller et al. (2019). no microaggressions Do not use microaggressions, i.e. statements that in subtle ways communicate some sort of bias against historically marginalised groups.”
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “[15] Luke Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov.”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “Since online toxicity can be relatively scarce (Founta et al., 2018),3 we start by annotating English Reddit posts, specifically three intentionally offensive subReddits and a corpus of potential microaggressions from Breitfeller et al. (2019).”
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2021, background): “Anti-Black language denotes racially prejudiced or racist content—subtle (Breitfeller et al., 2019) or overt—which is often a desired target for toxic language detection research (Waseem, 2016; Vid- 7For each post, we collected toxicity ratings from two white conservative workers, two from white…”
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021, positive): “Our work also complements recent efforts to capture and understand microaggressions (Breitfeller et al., 2019), a similarly elusive class that draws on subtle and unconscious linguistic reﬂections of social bias, prejudice and inequality (Sue, 2010).”
Case Study: Deontological Ethics in NLP (2020, background): “…the user would not know the answer to question in the first place, and so it would be difficult to determine even that the QA system was (a) Micro-aggressive comment and the score it received by existing state-of-the-art hate speech detection and sentiment analysis tools (Breitfeller et al., 2019).”
A Framework for the Computational Linguistic Analysis of Dehumanization (2020, background): “Language is central to studying this phenomenon; like other forms of bias (Wiebe et al., 2004; Greene and Resnik, 2009; Recasens et al., 2013; Voigt et al., 2017; Breitfeller et al., 2019), dehumanizing attitudes are expressed through subtle linguistic manipulations, even in carefullyedited texts.”
Documenting the English Colossal Clean Crawled Corpus (2021, background): “However, determining whether a document has toxic or lewd content is a more nuanced endeavor that goes beyond detecting “bad” words; hateful and lewd content can be expressed without negative keywords (e.g., mi-croaggressions, innuendos; Breitfeller et al., 2019; Dinan et al., 2019).”
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts (2021, methodology): “Existing datasets of offensive language (Breitfeller et al., 2019; Sap et al., 2020) annotated comments from potentially offensive SubReddits to increase proportion of offensive language.”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021, background): “Furthermore, in addition to abusive language [139] and hate speech [67], there are subtler forms of negativity such as gender bias [137], microaggressions [22], dehumanization [83], and various socio-political framing biases [44, 114] that are prevalent in language data.”
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (2022, background): “A broader deﬁnition would incorporate implicit toxicity, as when a speaker “subtly” or “unconsciously expresses a prejudiced attitude” [18, 70].”
Large Language Model Alignment: A Survey (2023, background): “may sometimes exhibit positive sentiment and is typically harder to detect or collect on a large scale (MacAvaney et al., 2019; Breitfeller et al., 2019).”
Leveraging Bias in Pre-Trained Word Embeddings for Unsupervised Microaggression Detection (2021, unknown): “com dataset (Breitfeller et al., 2019), comprised of 1,314 and 1,278 Tumblr posts respectively1.”
Towards Ethics by Design in Online Abusive Content Detection (2020, methodology): “Breitfeller et al. (2019) used the degree of discrepancies in annotations between male and female annotators to surface nuanced microag-gressions.”
Unsupervised Discovery of Implicit Gender Bias (2020, background): “This task is notoriously difficult because words like “pretty” often register as positive content, rather than as indicative of bias (Breitfeller et al., 2019; Jurgens et al., 2019).”
Introducing CAD: the Contextual Abuse Dataset (2021, methodology): “Breitfeller et al. (2019) use a collaborative multi-stage process to label micro-aggression and Card et al. (2015) use a similar process for labelling news articles.”