Racial categories in machine learning

By Sebastian Benthall, Bruce D. Haynes. 2018

141 citations recorded by the source corpus.

Controversies around race and machine learning have sparked debate among computer scientists over how to design machine learning systems that guarantee fairness. These debates rarely engage with how racial identity is embedded in our social experience, making for sociological and psychological complexity. This complexity challenges the paradigm of considering fairness to be a formal property of supervised learning with respect to protected personal attributes. Racial identity is not simply a personal subjective quality. For people labeled "Black" it is an ascribed political category that has consequences for social differentiation embedded in systemic patterns of social inequality achieved through both social and spatial segregation. In the United States, racial classification can best be understood as a system of inherently unequal status categories that places whites as the most privileged category while signifying the Negro/black category as stigmatized. Social stigma is reinforced through the unequal distribution of societal rewards and goods along racial lines that is reinforced by state, corporate, and civic institutions and practices. This creates a dilemma for society and designers: be blind to racial group disparities and thereby reify racialized social inequality by no longer measuring systemic inequality, or be conscious of racial categories in a way that itself reifies race. We propose a third option. By preceding group fairness interventions with unsupervised learning to dynamically detect patterns of segregation, machine learning systems can mitigate the root cause of social disparities, social segregation and stratification, without further anchoring status categories of disadvantage.

Subfields: Safety Research, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

Measurement and Fairness (2019, background): “ as \nancial stability," \mental health status," and \violent risk to society," become encoded in policy. Fundamentally, race as a category itself further entrenches structural racism [7, 27]. In computer science, it is particularly rare to articulate the distinctions between construc”
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing (2020, neutral): “a wide range of phenotypic features [3].”
Towards a critical race methodology in algorithmic fairness (2019, neutral): “Second, the critique levied by computational social scientist Sebastian Benthall and critical race scholar Bruce D. Haynes [12] describes how little attention has been given to the meaning of protected class labels, the political orientation of class labels, and the normative assumptions of machine ”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, methodology): “…data through inference and proxies (e.g., [32, 69, 90]), to only use demographic data during model training (e.g., [28, 49, 71, 88]) 1 , or to identify and mitigate algorithmic discrimination for computationally identified groups without the use of demographics at all (e.g., [13, 39, 53]).”
Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency (2021, background): “In addition, a more fundamental critique of racial and gender taxonomies is that they pose a risk of reifying racial and gender categories as natural and inherent rather than socially constructed [7, 34, 41].”
Addressing multiple metrics of group fairness in data-driven decision making (2020, background): “This points out to the fact that the categories considered in the dataset may yield diﬀerent results in terms of group fairness [5].”
How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, background): “Specifically, researchers have called for increased engagement with critical scholarship [64] and complex realities of identity [18, 150].”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, unknown): “Benthall and Haynes [11] proposed using unsupervised clustering to discover “race-like” categories, and Hanna et al. [56] levied a critique against such a mechanism.”
Fairness in Machine Learning: A Survey (2020, methodology): “Fair clustering has also seen applications in the discovery of potentially protected groups (e.g., References [21, 68]) and as a pre-processing method that augments the original data to achieve fairness [64].”
One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision (2021, methodology): “Our work is related in spirit to Hanna et.al, [18] as well as Ben-thall and Haynes [5], who both critique the usage of racial categories from the perspective of critical race theory.”
Stable Bias: Evaluating Societal Representations in Diffusion Models (2023, unknown): “…previous work in ML has adopted binary gender and fixed prior ethnicity categorizations for the sake of convenience, the downstream impact of these choices that are propagated from (labeled) datasets to trained models can contribute to perpetuating algorithmic harms and unfairness [7, 5, 31, 15].”
How Computers See Gender (2019, background): “Benthall and Haynes proposed an unsupervised training method for detecting racial segregation and using that information to mitigate racial biases in machine learning systems [12].”
Ethics of artificial intelligence and robotics (2020, background): “It appears that technological fixes have their limits in that they need a mathematical notion of fairness, which is hard to come by (Whittaker et al. 2018, 24ff; Selbst et al. 2019); as is a formal notion of ‘race’ (see Benthall and Haynes 2019).”
Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022, background): “Notably, even systems that seek to identify and mitigate racism, harassment and hate speech may compound harms to vulnerable people (e.g., [7, 14, 49]).”
Are My Deep Learning Systems Fair? An Empirical Study of Fixed-Seed Training (2021, background): “…software system for fairness (where models are continuously updated), such as identity-inference-based advertisement targeting systems [7, 70, 101], multiple identical runs are necessary, and statistical test results should be reported to measure the variance of bias values caused by…”