Towards a critical race methodology in algorithmic fairness

By A. Hanna, Emily L. Denton, Andrew Smart, Jamila Smith-Loud. 2019

342 citations recorded by the source corpus.

We examine the way race and racial categories are adopted in algorithmic fairness frameworks. Current methodologies fail to adequately account for the socially constructed nature of race, instead adopting a conceptualization of race as a fixed attribute. Treating race as an attribute, rather than a structural, institutional, and relational phenomenon, can serve to minimize the structural aspects of algorithmic unfairness. In this work, we focus on the history of racial categories and turn to critical race theory and sociological work on race and ethnicity to ground conceptualizations of race for fairness research, drawing on lessons from public health, biomedical research, and social survey research. We argue that algorithmic fairness researchers need to take into account the multidimensionality of race, take seriously the processes of conceptualizing and operationalizing race, focus on social processes which produce racial inequality, and consider perspectives of those most affected by sociotechnical systems.

Subfields: Genetics, Information Systems, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Information Systems

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “to their different social positions (Hanna et al., 2020)—particularly groups at the intersections of multiple axes of oppression—we recommend that researchers and practitioners center work analyzing “bias” in NLP systems around the lived”
Re-imagining Algorithmic Fairness in India and Beyond (2021, background): “However these categories are culturally and historically situated [85].”
Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “[29], we are actualizing a particular static conceptualization of observed race here.”
Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development (2021, neutral): “…design frameworks which have explicitly embraced specific politics and values [8, 24, 65, 81] Particularly with its focus on real-world applications, the artifacts produced by machine learning researchers and practitioners — for example, in the form of datasets — are value-laden [51, 98, 107].”
PaLM 2 Technical Report (2023, background): “We also note the challenges in design choices related to encoding and interpreting references to identity from automated systems (Hanna et al., 2020; Movva et al., 2023), and highlight this in the results related to "japanese people" (24%), "indian people" (19%) and "asian people" (6%).”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, background): “Benjamin, 2019; Buolamwini and Gebru, 2018; Lum and Isaac, 2016; Noble, 2018; Eubanks, 2018), exploitative or extractive data practices (Gray and Suri, 2019), and applications of AI that dispossess the identity and resources of vulnerable populations (Green, 2019; Keyes, 2018; Hanna et al., 2019; St”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “Which groups should be analyzed is application dependent [48] but, as practitioners may not have a speciﬁc deployment scenario in mind, it is worth discussing why particular groups and attributes are chosen for analysis, and the implications for interpreting results.”
A Survey of Race, Racism, and Anti-Racism in NLP (2021, background): “…and can refer to a variety of different perspectives, including racial identity (how you self-identify), observed race (the race others perceive you to be), and reﬂected race (the race you believe others perceive you to be) (Roth, 2016; Hanna et al., 2020; Ogbonnaya-Ogburu et al., 2020).”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Additionally, the deﬁnition of racial and ethnic groups has ﬂuctuated across time and place (Hanna et al. 2020).”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “Work in this area has addressed race [13, 35], gender [34, 43, 72], and disability [11], calling into question what it means to even rely on these categories as a basis for assessing unfairness, and what harms are reproduced by relying on these infrastructures of categorization.”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “8 The foundation of this problem is that categorizing people into discrete, socially constructed groups, while tenuous, is often necessary for machine learning systems to make sense of socially relevant distinctions [35, 56, 67].”
Towards Measuring Fairness in AI: The Casual Conversations Dataset (2021, background): “On the other hand, Hanna et al. [36] emphasizes that race is socially constructed and in algorithmic fairness methodologies ML practitioners should be mindful of the dimensions of race that are relevant for particular contexts.”
The Use and Misuse of Counterfactuals in Ethical Machine Learning (2021, background): “[23] use critical race theory and argue that the multi-dimensionality of race should be taken into account whenever this phenomenon becomes relevant to the machine learning community, and challenges practitioners to explicitly ask who is doing the categorizing and for what purpose? Barocas et al.”
The uselessness of AI ethics (2022, background): “These are complex questions which have shifted substantially over time and which intersect with race, gender, and culture [29, 58].”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, methodology): “…dynamic and non-discrete concepts such as gender and using skin tone as a proxy for race, we reify certain conceptions of these concepts (Hanna et al., 2020; Jacobs and Wallach, 2021) that harm certain groups, e.g., non-binary individuals (Hamidi et al., 2018; Scheuerman et al., 2020).”