Measurement and Fairness

By Abigail Z. Jacobs, Hanna M. Wallach. 2019

535 citations recorded by the source corpus.

We propose measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems. Computational systems often involve unobservable theoretical constructs, such as socioeconomic status, teacher effectiveness, and risk of recidivism. Such constructs cannot be measured directly and must instead be inferred from measurements of observable properties (and other unobservable theoretical constructs) thought to be related to them---i.e., operationalized via a measurement model. This process, which necessarily involves making assumptions, introduces the potential for mismatches between the theoretical understanding of the construct purported to be measured and its operationalization. We argue that many of the harms discussed in the literature on fairness in computational systems are direct results of such mismatches. We show how some of these harms could have been anticipated and, in some cases, mitigated if viewed through the lens of measurement modeling. To do this, we contribute fairness-oriented conceptualizations of construct reliability and construct validity that unite traditions from political science, education, and psychology and provide a set of tools for making explicit and testing assumptions about constructs and their operationalizations. We then turn to fairness itself, an essentially contested construct that has different theoretical understandings in different contexts. We argue that this contestedness underlies recent debates about fairness definitions: although these debates appear to be about different operationalizations, they are, in fact, debates about different theoretical understandings of fairness. We show how measurement modeling can provide a framework for getting to the core of these debates.

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “…al., 2021b), there is a lack of standardization of fairness benchmarks, an understanding of what harms different bias measures in NLP relate to (Blodgett et al., 2020, 2021; Jacobs & Wallach, 2021), and coverage of identities in fluid, comprehensive ways (Cao & Daumé III, 2020; Dev et al., 2021a).”
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, background): “, karma thresholding, filtering out specific slurs and swearwords) biases the language model’s pretraining data and behavior (Jacobs and Wallach, 2019).”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “…been acknowledged broadly by the machine learning community as a particularly difﬁcult task in benchmark design and development [Mitchell, 2021, Jacobs and Wallach, 2021, Malik, 2020] — however as Bowman and Dahl note, it remains a generally neglected issue as “[t]his criterion is difﬁcult to…”
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, background): “However, in many scenarios, the labels may be inaccurate, either due to measurement issues (Jacobs and Wallach 2021) or due to discrimina-”
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets (2021, methodology): “Measurement Modeling We apply a measurement modeling lens by viewing each benchmark as a measurement model (MM) (e.g., Quinn et al., 2010; Jacobs and Wallach, 2021).”
OPT: Open Pre-trained Transformer Language Models (2022, background): “While there may be shortcomings in these benchmarks (Blodgett et al., 2021; Jacobs and Wallach, 2021), these measurements provide a first step towards understanding the limitations of OPT-175B.”
GLM-130B: An Open Bilingual Pre-trained Model (2022, background): “Notwithstanding their deficiency (Blodgett et al., 2021; Jacobs & Wallach, 2021), these datasets serve as a meaningful initial step towards an open quantitative evaluation LLMs.”
Understanding and Evaluating Racial Biases in Image Captioning (2021, neutral): “This is consistent with the argument of Jacobs and Wallach [37]: gender is operationalized differently in caption-derived versus human-collected annotations.”
Measuring Data (2022, background): “As such, it is important not to abstract the actual methods that are used to derive a measurement and to directly reference deﬁnitions and explain any assumptions used to perform a calculation up front (Jacobs and Wallach, 2021).”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, background): “However, the mismatch between the construct of toxicity and its operationalization through an automatic classi-ﬁer can cause biased or unintended model behavior (Jacobs and Wallach, 2021).”
Towards a critical race methodology in algorithmic fairness (2019, background): “hich stem from bureaucratic processes. We know from thehistoriesoutlinedabovethatthesecategoriesareunstable,contingent, and rooted in racial inequality. Followingrecentworkaroundmeasurementandfairness[5,57], we suggest taking seriously the problems of measurement modeling when considering racial var”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “Moreover, bias measures may cause further epistemic violence onto the marginalized by creating a veneer of fairness, in spite of ongoing marginalization (Gonen and Goldberg, 2019; Talat et al., 2021; Jacobs and Wallach, 2021).”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Differentially private fair machine learning (Jagielski et al. 2019) could potentially provide such guarantees, simultaneously meeting the requirements of fairness, privacy and accuracy.”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “Although the algorithmic fairness literature has frequently called out the dangers of using proxies for salient attributes or target variables in fairness analyses [9, 19, 45], this is often the only option practitioners have to make a quantitative argument for escalation and deeper analysis.”
Gender Bias in Machine Translation (2021, background): “For a broader discussion on measurement and bias we refer the reader also to (Jacobs, 2021; Jacobs et al., 2020).”