50 Years of Test (Un)fairness: Lessons for Machine Learning

By Ben Hutchinson, Margaret Mitchell. 2018

401 citations recorded by the source corpus.

Quantitative definitions of what is unfair and what is fair have been introduced in multiple disciplines for well over 50 years, including in education, hiring, and machine learning. We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged. In some cases, earlier definitions of fairness are similar or identical to definitions of fairness in current machine learning research, and foreshadow current formal work. In other cases, insights into what fairness means and how to measure it have largely gone overlooked. We compare past and current notions of fairness along several dimensions, including the fairness criteria, the focus of the criteria (e.g., a test, a model, or its use), the relationship of fairness to individuals, groups, and subgroups, and the mathematical method for measuring fairness (e.g., classification, regression). This work points the way towards future research and measurement of (un)fairness that builds from our modern understanding of fairness while incorporating insights from the past.

Subfields: Safety Research, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

Measurement and Fairness (2019, background): “, [5, 8, 13, 33]), all use very stylized models of fairness from which the language of measurement is largely, and crucially, missing.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Moverover, as geographic and cultural contexts shift, so do the meaning of different categories [206, 112] and must be interpreted according to their local meaning.”
Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy (2019, background): “A great overview of the history of machine learning fairness can be found in Hutchinson and Mitchell [38].”
A Survey on Bias and Fairness in Machine Learning (2019, background): “rvey on Bias and Fairness in Machine Learning 11 4.2 Definitions of Fairness In [16], authors studied fairness definitions in political philosophy and tried to tie them to machinelearning. Authors in [60] studied the 50-year history of fairness definitions in the areas of education and machine-learn”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “For example, since the 1960s, psychometricians have studied the fairness of educational tests based on their ability to predict performance (at school or work) [20, 21, 121, 28, 36, 106, 60, 79, 61].”
Mitigating bias in algorithmic hiring: evaluating claims and practices (2019, background): “This has led to a decades-long line of literature seeking to measure and mitigate the observed disparities (see [50] for a survey).”
AI for social good: unlocking the opportunity for positive impact (2020, methodology): “Fairness in particular should be explicitly accounted for, to avoid reinforcing existing societal biases re ﬂ ected in the data used for model development 45 – 47 .”
Critical Perspectives on Governance Mechanisms for AI/ML Systems (2020, background): “These efforts have historical parallels in the social sciences, particularly around quantitative educational, vocational (Hutchinson & Mitchell, 2019), and psychometric testing (Lussier, 2018).”
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2020, background): “The coreference task (Figure 3, rows 1-3) doesn’t display a clear correlation in all cases, and yet it has the clearest relationship of all three tasks, with a signiﬁcant moderate positive correlation for both Predictive Parity (precision) and Equality of Opportunity (re-call) for word2vec (columns ”
Awareness in practice: tensions in access to sensitive attribute data for antidiscrimination (2019, background): “ork The fair ML research community has long reflected on the social and policy contexts of its work, recognizing legal tensions [8], historical parallels in prior debates over definitions of fairness [37], and the limitations of data-dependent problem formulation [64, 70]. However, when emphasizing ”
On the apparent conflict between individual and group fairness (2019, background): “p fairness measure. 1Formalised deﬁnitions of fairness which foreshadow those in Fair-ML did appear in areas other than ML, such as testing in education and hiring. See Hutchinson & Mitchell 2019 [30]. On the Apparent Conflict Between Individual and Group Fairness FAT* ’20, January 27–30, 2020, ”
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023, background): “…Fairness Previous research has shown that the performance of models for downstream tasks can vary greatly among different identity groups (Hovy and Søgaard, 2015; Buolamwini and Gebru, 2018; Dixon et al., 2018), highlighting the issue of fairness (Hutchinson and Mitchell, 2019; Liu et al., 2020).”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, background): “Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”