Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation

By Angelina Wang, Vikram V. Ramaswamy, Olga Russakovsky. 2022

129 citations recorded by the source corpus.

Research in machine learning fairness has historically considered a single binary demographic attribute; however, the reality is of course far more complicated. In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups. For each question, we provide thorough empirical evaluation on tabular datasets derived from the US Census, and present constructive recommendations for the machine learning community. First, we advocate for supplementing domain knowledge with empirical validation when choosing which demographic attribute labels to train on, while always evaluating on the full set of demographic attributes. Second, we warn against using data imbalance techniques without considering their normative implications and suggest an alternative using the structure in the data. Third, we introduce new evaluation metrics which are more appropriate for the intersectional setting. Overall, we provide substantive suggestions on three necessary (albeit not sufficient!) considerations when incorporating intersectionality into machine learning.

Subfields: Health Informatics, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Informatics

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Many evaluations focus on distinct or binary groups, due to the complexity of operationalising intersectionality [257, 133]; in many cases, assumptions used to simplify for the sake of mathematical notation and interpretation result in obscuring the very phenomena they seek to describe [64].”
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (2023, background): “However, recent studies in NLP fairness (Wang et al., 2022a; Ovalle et al., 2023b) points out that some AI fairness literature fail to discuss the source of biases investigated, framing AI fairness as solely a technical problem.”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “While computing literature often describes stereotyping along single-axis dimensions of identity, an intersectional approach draws attention to how harms play out for people whose lives are shaped by interlocking forms of oppression [203] – for example, when a search for the term “unprofessional hai”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “…244], an intersectional approach draws attention to how harms play out for people whose lives are shaped by interlocking forms of oppression [46, 229] – for aim(s)tocapturethemor-phological sexual differences between male and female faces by comparing their shape differences to a defined face…”