Does mitigating ML's disparate impact require disparate treatment?

By Zachary Chase Lipton, Alexandra Chouldechova, Julian McAuley. 2017

17 citations recorded by the source corpus.

Following related work in law and policy, two notions of prejudice have come to shape the study of fairness in algorithmic decision-making. Algorithms exhibit disparate treatment if they formally treat people differently according to a protected characteristic, like race, or if they intentionally discriminate (even if via proxy variables). Algorithms exhibit disparate impact if they affect subgroups differently. Disparate impact can arise unintentionally and absent disparate treatment. The natural way to reduce disparate impact would be to apply disparate treatment in favor of the disadvantaged group, i.e. to apply affirmative action. However, owing to the practice's contested legal status, several papers have proposed trying to eliminate both forms of unfairness simultaneously, introducing a family of algorithms that we denote disparate learning processes (DLPs). These processes incorporate the protected characteristic as an input to the learning algorithm (e.g.~via a regularizer) but produce a model that cannot directly access the protected characteristic as an input. In this paper, we make the following arguments: (i) DLPs can be functionally equivalent to disparate treatment, and thus should carry the same legal status; (ii) when the protected characteristic is redundantly encoded in the nonsensitive features, DLPs can exactly apply any disparate treatment protocol; (iii) when the characteristic is only partially encoded, DLPs may induce within-class discrimination. Finally, we argue the normative point that rather than masking efforts towards proportional representation, it is preferable to undertake them transparently.

Subfields: Safety Research, Statistics and Probability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Statistics and Probability

In AI Ethics, Fairness & Society

Citations in the corpus

Blind Justice: Fairness with Encrypted Sensitive Attributes (2018, background): “In order to check and enforce such requirements, the modeler must have access to the sensitive attributes in the training data (Žliobaitė & Custers, 2016; Lipton et al., 2017)— however, this may be undesirable for several reasons.”
Algorithmic Fairness (2020, background): “A theoretical analysis of the trade-off between fairness and accuracy was studies in [39] and [91].”
Translation Tutorial : A Shared Lexicon for Research and Practice in Human-Centered Software Systems (2018, background): “This view is carefully laid out by Nissenbaum (1996), who argues that such responsibility is held back in computer systems by a number of factors related to the way these systems are designed, implemented, and fielded.”
Auditing Black-Box Models Using Transparent Model Distillation With Side Information (2017, background): “See [32] for a discussion of these concepts in the context of machine learning models.”