The Use and Misuse of Counterfactuals in Ethical Machine Learning

By Atoosa Kasirzadeh, Andrew Smart. 2021

110 citations recorded by the source corpus.

The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be considered are social categories such as race or gender. We review a broad body of papers from philosophy and social sciences on social ontology and the semantics of counterfactuals, and we conclude that the counterfactual approach in machine learning fairness and social explainability can require an incoherent theory of what social categories are. Our findings suggest that most often the social categories may not admit counterfactual manipulation, and hence may not appropriately satisfy the demands for evaluating the truth or falsity of counterfactuals. This is important because the widespread use of counterfactuals in machine learning can lead to misleading results when applied in high-stakes domains. Accordingly, we argue that even though counterfactuals play an essential part in some causal inferences, their use for questions of algorithmic fairness and social explanations can create more problems than they resolve. Our positive result is a set of tenets about using counterfactuals for fairness and explanations in machine learning.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

The Capacity for Moral Self-Correction in Large Language Models (2023, background): “decision making systems [5], in which this type of evaluation is more common [14, 30], though not without pitfalls that also apply to our work [26].”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “Their permissibility of use, however, for addressing class imbalance has been contested because of the infeasibility of manipulating demographic categories [62, 74, 110] and inaccurate conceptualizations of causality of demographic categories [64, 83].”
Counterfactual Fairness Is Basically Demographic Parity (2022, background): “This individual notion of fairness promises to avoid the pitfalls of the group based fairness measures we described earlier, though the relationship between causality, counterfactuals and moral philosophy is complicated (Kasirzadeh and Smart 2021).”