The Problem of Infra-marginality in Outcome Tests for Discrimination

By Camelia Simoiu, Sam Corbett-Davies, Sharad Goel. 2016

196 citations recorded by the source corpus.

Outcome tests are a popular method for detecting bias in lending, hiring, and policing decisions. These tests operate by comparing the success rate of decisions across groups. For example, if loans made to minority applicants are observed to be repaid more often than loans made to whites, it suggests that only exceptionally qualified minorities are granted loans, indicating discrimination. Outcome tests, however, are known to suffer from the problem of infra-marginality: even absent discrimination, the repayment rates for minority and white loan recipients might differ if the two groups have different risk distributions. Thus, at least in theory, outcome tests can fail to accurately detect discrimination. We develop a new statistical test of discrimination---the threshold test---that mitigates the problem of infra-marginality by jointly estimating decision thresholds and risk distributions via a hierarchical Bayesian latent variable model. Applying our test to a dataset of 4.5 million police stops in North Carolina, we find that the problem of infra-marginality is more than a theoretical possibility, and can cause the outcome test to yield misleading results in practice.

Subfields: Sociology and Political Science, Statistics and Probability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Sociology and Political Science

In Statistics and Probability

In AI Ethics, Fairness & Society

Citations in the corpus

Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, methodology): “This definition of fairness is also sometimes called called predictive parity [12], and it is assessed by an outcome test [115].”
50 Years of Test (Un)fairness: Lessons for Machine Learning (2018, background): “t we know of. There are conceptual similarities between Jones’ criteria and the constrained ranking problem described by [8], and also between Einhorn’s criterion and concerns about infra-marginality [60]. For a binary classifier, Thorndike’s 1971 group parity criterion is equivalent to requiring th”
Algorithmic Decision Making and the Cost of Fairness (2017, background): “…blue distribution, resulting in higher detention and false positive rates in the red group. a single decision threshold can, and generally does, result in racial disparities is closely related to the notion of infra-marginality in the econometric literature on taste-based discrimination [3, 4, 34].”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, methodology): “This definition of fairness is also some-times called predictive parity (Chouldechova 2017), and it is assessed by an outcome test (Simoiu et al. 2017).”
On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning (2018, background): “These issues have motivated a growing stream of technical work on disparate impact in ML, focusing on topics such as: (i) how to identify and quantify disparate impact [8, 9, 10, 11]; (ii) how to train models that mitigate disparate impact [12, 13, 14]; and (iii) how to identify causal factors of di”
Bayesian Modeling of Intersectional Fairness: The Variance of Bias (2018, background): “…(cid:101) 1 ) - DF bias ampliﬁcation is a more politically conservative fairness metric which does not seek to correct unfairness in the original dataset, and reﬂects the principle of inframarginal-ity ( a system is biased only if disparities in its behavior are worse than those in society ) [29].”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Some researchers use structural assumptions to infer rank orders or thresholds (Simoiu et al., 2017).”
Race Discrimination: An Economic Perspective (2020, background): “Simoiu, Corbett-Davies, and Goel (2017) model the police decision to search as a function of a continuous signal, sent by drivers to police, on their likelihood of carrying contraband.”
A General Approach to Fairness with Optimal Transport (2020, background): “Whilst facilitating model evaluation and design, not accounting for the full shapes of the relevant distributions can be restrictive and problematic (Simoiu, Corbett-Davies, and Goel 2017).”
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “…disparate impact arises (Chen et al., 2018; Datta et al., 2016; Adler et al., 2018), how it can be measured (Žliobait˙e, 2017; Pierson et al., 2017; Simoiu et al., 2017; Kilbertus et al., 2017; Kusner et al., 2017; Galhotra et al., 2017), and how it can be mitigated (Feldman et al., 2015;…”
Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019, background): “Demographic Parity (DP) [6, 13, 22, 47, 59], requires an algorithm to classify both the protected and the privileged group with the same probability.”
An Intersectional Definition of Fairness (2018, background): “ted with race [21]. On the other hand, attributes which are important for the algorithm may have different distributions depending on the protected attributes; a phenomenon known as infra-marginality [32]. Consequently, equity may not always correspond to fairness, as deﬁned by [34]. It is also impo”
Fast Threshold Tests for Detecting Discrimination (2017, background): “On all three measures, our new threshold model generally outperforms that of Simoiu et al. (2017); the improvement in gradient computation is partic-2 ularly substantial.”
Fairness Definitions Explained (2018, unknown): “acceptance rate [24], benchmarking [9]).”