Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions

By Shira Mitchell, Eric Potash, Solon Barocas. 2018

182 citations recorded by the source corpus.

A recent flurry of research activity has attempted to quantitatively define "fairness" for decisions based on statistical and machine learning (ML) predictions. The rapid growth of this new field has led to wildly inconsistent terminology and notation, presenting a serious challenge for cataloguing and comparing definitions. This paper attempts to bring much-needed order. First, we explicate the various choices and assumptions made---often implicitly---to justify the use of prediction-based decisions. Next, we show how such choices and assumptions can raise concerns about fairness and we present a notationally consistent catalogue of fairness definitions from the ML literature. In doing so, we offer a concise reference for thinking through the choices, assumptions, and fairness considerations of prediction-based decision systems.

Subfields: Artificial Intelligence, Safety Research, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

Model Cards for Model Reporting (2018, background): “rding to the chosen metrics, providing confidence interval values when possible. Parity on the different metrics across disaggregated population subgroups corresponds to how fairness is often defined [37, 48]. Quantitative analyses should demonstrate the metric variation (e.g., with error bars), as ”
Measurement and Fairness (2019, background): “, [46])—will help assess these systems in practice.”
A Causal Bayesian Networks Viewpoint on Fairness (2018, neutral): “This eﬀort has led to the emergence of a fairness taxonomy [12, 14, 20, 23] providing researchers and practitioners with diagnostics tools to evaluate existing systems or with constraints when designing new ones.”
Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, background): “Our focus on differences in average predictions across groups is a common relaxation of parity-based predictive fairness definitions (Corbett-Davies et al., 2017; Mitchell et al., 2019).”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, background): “Such questions speak to a critical practice whose recent efforts, in response, have proposed fairness metrics that attempt to use causality (Chiappa and Isaac, 2018; Mitchell et al., 2018; Nabi and Shpitser, 2018; Madras et al., 2019) or interactivity (Canetti et al.”
Perturbation Sensitivity Analysis to Detect Unintended Model Biases (2019, background): “Second, a range of criteria deﬁne unfairness for groups , each in terms of statistical dependence between group membership, model score, and class label (see, e.g., (Choulde-chova and Roth, 2018; Mitchell et al., 2018)).”
Counterfactual risk assessments, evaluation, and fairness (2019, background): “1This distinction is also made in a survey of fairness literature [34].”
An Intersectional Definition of Fairness (2018, background): “The necessity has now become clear for interdisciplinary analyses of fairness in AI and its relationship to society, to civil rights, and to the social goals which are to be achieved by mathematical fairness definitions, which have not always been made explicit [18].”
Average Individual Fairness: Algorithms, Generalization and Experiments (2019, background): “The literature on fairness in machine learning has become much too large to comprehensively summarize, but see Mitchell et al. (2018) for a recent survey.”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “The first of three core challenges in incorporating intersectionality that we address in this work is considering which identities to include [95].”
This Thing Called Fairness (2019, background): “conferences [13, 57, 108] as well as excellent summaries in [103, 104, 148].”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “Yet, these mathematical and computational approaches do not adequately capture the societal impact of AI systems [62, 74, 76, 113].”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Mitchell et al. (2018) catalogues the choices made by software engineers as they build models.”
A Framework for Understanding Unintended Consequences of Machine Learning (2019, background): “Section 4.4 of Mitchell et al. (2018) further discusses different metrics.”
Fairness Violations and Mitigation under Covariate Shift (2019, background): “A number of fairness metrics have been proposed that make different normative statements on the machine learning models’ output (see [37] for a review).”