The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning

By Sam Corbett-Davies, Sharad Goel. 2018

907 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Measurement and Fairness (2019, background): “This nontrivial measurement task is deeply complex and requires significant debate about justice, policy, and fairness [8, 13, 15, 37].”
Fair Regression: Quantitative Definitions and Reduction-based Algorithms (2019, background): “Copyright 2019 by the author(s). have been proposed (Berk et al., 2017; Chouldechova, 2017; Hardt et al., 2016; Kleinberg et al., 2017), leading to a variety of algorithms that aim to satisfy them (see, e.g., Corbett-Davies & Goel, 2018, for an overview of the literature).”
Efficient candidate screening under multiple tests and implications for fairness (2019, neutral): “) [6] or assessing some penalty (e.”
A Causal Bayesian Networks Viewpoint on Fairness (2018, neutral): “Many such definitions have been found to be mathematically incompatible [7,13,15,16,30], and this has been viewed as representing an unavoidable trade-off establishing fundamental limits on fair machine learning, or as an indication that certain definitions do not map on to social or legal understan”
Towards a critical race methodology in algorithmic fairness (2019, neutral): “While limitations of group fairness criteria have been previously explored [26, 47], here we focus our examination on the conceptualization of race embedded within this framework.”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “Classical fairness metrics [25, 29, 75] usually require specifying a protected attribute, such as sexual orientation or race.”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “2 For example, classification parity approaches to fairness aim to equalise predictive performance measures across groups, whereas anti-classification parity approaches rely on the omission of protected attributes from the decision making process to ensure different groups receive equivalent treatme”
Algorithmic Fairness (2020, background): “Interestingly, several recent studies have shown that it is not possible to satisfy multiple notions of fairness simultaneously [15, 36, 38, 39, 56, 85, 112].”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “Although the algorithmic fairness literature has frequently called out the dangers of using proxies for salient attributes or target variables in fairness analyses [9, 19, 45], this is often the only option practitioners have to make a quantitative argument for escalation and deeper analysis.”
Fairness without Harm: Decoupled Classifiers with Preference Guarantees (2019, background): “In addition to Zafar et al. (2017b) and Dwork et al. (2018), several other works discuss treatment disparity to achieve parity-based notions of fairness (see e.g., Corbett-Davies & Goel, 2018; Kleinberg et al., 2018; Lipton et al., 2018; Wang et al., 2019).”
The Implicit Fairness Criterion of Unconstrained Learning (2018, background): “For a more complete treatment of algorithmic fairness literature, the reader is referred to Barocas et al. (2018); Chouldechova and Roth (2018); Corbett-Davies and Goel (2018).”
Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies (2023, background): “Corbett-Davies and collaborators introduced risk-minimization approaches aimed at minimizing disparities [27,28].”
Towards causal benchmarking of bias in face analysis algorithms (2020, background): “The machine learning community is active in analyzing biases of learning models, and how one may train models where bias is mitigated [3, 14, 18, 31, 33, 41, 46, 51, 68], usually by ensuring that performance is equal across certain subgroups of a dataset.”