Fairness in Decision-Making - The Causal Explanation Formula

By Junzhe Zhang, Elias Bareinboim. 2018

326 citations recorded by the source corpus.

AI plays an increasingly prominent role in society since decisions that were once made by humans are now delegated to automated systems. These systems are currently in charge of deciding bank loans, criminals' incarceration, and the hiring of new employees, and it's not difficult to envision that they will in the future underpin most of the decisions in society. Despite the high complexity entailed by this task, there is still not much understanding of basic properties of such systems. For instance, we currently cannot detect (neither explain nor correct) whether an AI system can be deemed fair (i.e., is abiding by the decision-constraints agreed by society) or it is reinforcing biases and perpetuating a preceding prejudicial practice. Issues of discrimination have been discussed extensively in political and legal circles, but there exists still not much understanding of the formal conditions that a system must meet to be deemed fair. In this paper, we use the language of structural causality (Pearl, 2000) to fill in this gap. We start by introducing three new fine-grained measures of transmission of change from stimulus to effect, which we called counterfactual direct (Ctf-DE), indirect (Ctf-IE), and spurious (Ctf-SE) effects. We then derive what we call the causal explanation formula, which allows the AI designer to quantitatively evaluate fairness and explain the total observed disparity of decisions through different discriminatory mechanisms. We apply these measures to various discrimination analysis tasks and run extensive simulations, including detection, evaluation, and optimization of decision-making under fairness constraints. We conclude studying the trade-off between different types of fairness criteria (outcome and procedural), and provide a quantitative approach to policy implementation and the design of fair AI systems.

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science, Causal Inference & Discovery, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Sociology and Political Science

In Causal Inference & Discovery

In AI Ethics, Fairness & Society

Citations in the corpus

Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “”(4) Zhang and Bareinboim decompose the total disparity into disparities from each type of path (direct, indirect, back-door) [133].”
Learning Optimal Fair Policies (2018, background): “…et al. 2015; Hardt, Price, and Srebro 2016; Kamiran, Zliobaite, and Calders 2013; Corbe-Davies et al. 2017; Jabbari et al. 2017; Kusner et al. 2017; Zhang and Bareinboim 2018; Mitchell and Shalden 2018; Zhang, Wu, and Wu 2017), so far meth-ods for optimal policy learning subject to fairness…”
A Causal Bayesian Networks Viewpoint on Fairness (2018, neutral): “3, in this section we further investigate unfairness in a dataset ∆ = {a, x, y}n=1, discuss issues that might arise when building a decision system from it, and show how to measure and deal with unfairness in complex scenarios, revisiting and extending material from [11,33,48].”
Causal Reasoning for Algorithmic Fairness (2018, background): “This work also illustrates alternative interpretations of causal unfairness: in our work, we would not consider Ŷ to be unfair if the causal graph is A ← X → Ŷ , as A is not a cause of Ŷ , while [44] would label it as a type of “spurious” discrimination.”
Path-Specific Counterfactual Fairness (2018, methodology): “…et al., 2017; Kilbertus et al., 2017; Kusner et al., 2017; Nabi & Sh-pitser, 2018; Russell et al., 2017; Zhang et al., 2017; Zhang & Wu, 2017; Zhang & Bareinboim, 2018) that using a causal framework (Pearl, 2000; Dawid, 2007; Pearl et al., 2016; Peters et al., 2017) would lead to a more…”
Towards Causal Representation Learning (2021, neutral): “include robust, explainable, or fair [142, 133, 276, 130, 260].”
On the Fairness of Causal Algorithmic Recourse (2020, background): “…a protected attribute on the prediction, thus making fairness a fundamentally causal problem (Kilbertus et al. 2017; Russell et al. 2017; Loftus et al. 2018; Zhang and Bareinboim 2018a,b; Nabi and Shpitser 2018; Nabi, Malinsky, and Shpitser 2019; Chiappa 2019; Salimi et al. 2019; Wu et al. 2019).”
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, background): “The importance of causal reasoning for an ethical and socially beneficial use of ML-assisted technology has also been stressed in a number of recent works in the field of explainability and fair algorithmic decision making [29, 42, 24, 63, 64, 10, 57, 15].”
Causality for Machine Learning (2019, background): “modern machine learning to learn meaningful representations for data, where meaningful can mean robust, transferable, interpretable, explainable, or fair (Kusner et al., 2017; Kilbertus et al., 2017; Zhang and Bareinboim, 2018). To combine structural causal modeling (1) and representation learning, ”
Estimation Considerations in Contextual Bandits (2017, methodology): “he methods proposed in Section2maybeusedtoimprovelearningperformance. 3.1 RegretinthePresenceofBiasedTrainingData Inrecentyears,therehasbeenincreasedawarenessaroundtheissueoffairnessinmachine learning[48]. Acontextualbanditattemptingtolearnapersonalizedpolicyissusceptible toprejudicepresentinthetrai”
Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, background): “SIGKDD Explorations Volume 22, Issue 1 33 [110] J. Zhang and E. Bareinboim.”
Algorithmic Fairness (2020, neutral): “explaining and quantifying discrimination [146].”
Meaningful Explanations of Black Box AI Decision Systems (2019, methodology): “Such tools have not been applied speciﬁcally to explain ML models, with a few exceptions dealing on speciﬁc issues, such as discrimination inference (Zhang and Bareinboim 2018; Bonchi et al. 2017; Caliskan, Bryson, and Narayanan 2017; Caravagna et al. 2016) and reasoning on sample selection bias…”
The Sensitivity of Counterfactual Fairness to Unmeasured Confounding (2019, background): “…causal models, and (b) using causal primitives such as interventions and counterfactuals to formalize “similar individuals”, causality provides valuable insights to resolve these conceptual roadblocks (Kil-bertus et al., 2017; Kusner et al., 2017; Nabi & Shpitser, 2018; Zhang & Bareinboim, 2018).”
Evaluating Explanations by Cognitive Value (2018, unknown): “In particular, the authors in [5] deﬁne the concepts of counterfactual direct eﬀect, counterfactual indirect eﬀect and counterfactual spurious eﬀects in order to estimate the discover discrimination and argue that by disentangling each of the causal eﬀects, it can be ascertained whether there was…”
Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “Inspired by the success of causal modeling [166], studying fairness from the causal perspective [135], [136], [137], [138], [139] has attracted increasing attentions.”