Exposing the probabilistic causal structure of discrimination

By Francesco Bonchi, Sara Hajian, B. Mishra, Daniele Ramazzotti. 2015

78 citations recorded by the source corpus.

Discrimination discovery from data is an important data mining task, whose goal is to identify patterns of illegal and unethical discriminatory activities against protected-by-law groups, e.g., ethnic minorities. While any legally valid proof of discrimination requires evidence of causality, the state-of-the-art methods are essentially correlation based, albeit, as it is well known, correlation does not imply causation. In this paper, we take a principled causal approach to discrimination detection following Suppes’ probabilistic causation theory. In particular, we define a method to extract, from a dataset of historical decision records, the causal structures existing among the attributes in the data. The result is a type of constrained Bayesian network, which we dub Suppes-Bayes causal network (SBCN). Next, we develop a toolkit of methods based on random walks on top of the SBCN, addressing different anti-discrimination legal concepts, such as direct and indirect discrimination, group and individual discrimination, genuine requirement, and favoritism. Our experiments on real-world datasets confirm the inferential power of our approach in all these different tasks.

Subfields: Management of Technology and Innovation, Safety Research, Sociology and Political Science, Causal Inference & Discovery, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management of Technology and Innovation

In Safety Research

In Sociology and Political Science

In Causal Inference & Discovery

In AI Ethics, Fairness & Society

Citations in the corpus

Causal Modeling-Based Discrimination Discovery and Removal: Criteria, Bounds, and Algorithms (2019, methodology): “In [23], the authors proposed a framework based on the SuppesBayes causal network and developed several random-walkbased methods to detect different types of discrimination.”
Path-Specific Counterfactual Fairness (2018, methodology): “Among them, the causal framework (Pearl 2000; Dawid 2007; Pearl, Glymour, and Jewell 2016; Peters, Janzing, and Schölkopf 2017) offers an intuitive and powerful way of reasoning about fairness, by viewing unfairness as the presence of an unfair causal effect of the sensitive attribute on the decisio”
Avoiding Discrimination through Causal Reasoning (2017, background): “Moreover, causality has already been employed for the discovery of discrimination in existing data sets by [14, 17].”
A Causal Framework for Discovering and Removing Direct and Indirect Discrimination (2016, methodology): “In (Bonchi et al. 2015), the authors proposed a framework based on the Suppes-Bayes causal network and developed several random-walk-based methods to detect di ﬀ erent types of discrimination.”
Fairness in Decision-Making - The Causal Explanation Formula (2018, background): “There is a growing literature in AI that is concerned with issues of transparency and fairness following, more or less explicitly, these two legal frameworks, including (not exhaustively) (Dwork et al. 2012; Romei and Ruggieri 2014; Mancuhan and Clifton 2014; Datta, Sen, and Zick 2016; Barocas and S”
Using Loglinear Model for Discrimination Discovery and Prevention (2016, methodology): “The authors in [17] proposed the use of SuppesBayes causal network and developed several random-walkbased methods over the causal structure to detect different types of discrimination.”
Meaningful Explanations of Black Box AI Decision Systems (2019, methodology): “Such tools have not been applied specifically to explain ML models, with a few exceptions dealing on specific issues, such as discrimination inference (Zhang and Bareinboim 2018; Bonchi et al. 2017; Caliskan, Bryson, and Narayanan 2017; Caravagna et al. 2016) and reasoning on sample selection bias (”
Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction (2018, background): “Our observations have important implications for recent proposals to avoid algorithmic discrimination through causal reasoning [8, 34, 39, 42, 51, 54, 63].”
Anti-discrimination learning: a causal modeling-based framework (2017, methodology): “Studies in [38–42] are built on causal modeling and the associated causal inference techniques, and the study in [4] is based on the Suppes–Bayes causal network and random-walk-based methods.”
On Discrimination Discovery Using Causal Networks (2016, methodology): “in [1] proposed a random walk method based on the Suppes-Bayes causal network.”
Causal Discrimination Discovery Through Propensity Score Analysis (2016, background): “The discrimination-aware data mining community has recently recognized the importance of causal analysis and the perils of correlation analysis [3, 6] while causation has been recognized as significant in the legal circles for a while [10].”
FA*IR: A Fair Top-k Ranking Algorithm (2017, background): “ future work. 2 RELATED WORK Anti-discrimination has only recently been considered from an algorithmic perspective [15]. Some proposals are oriented to discovering and measuring discrimination (e.g., [1, 3, 31]); while others deal with mitigating or removing discrimination (e.g., [10, 16, 17, 21, 38”
Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings (2019, methodology): “A nuanced account of the situation can be obtained using causal graphs (Bonchi et al., 2017; Chiappa, 2019; Kilbertus et al., 2017; Kusner et al., 2017; Mancuhan and Clifton, 2014; Pearl, 2009; Zhang and Wu, 2017).”
Measuring discrimination in algorithmic decision making (2017, methodology): “…a standard model and then modiﬁes this model to obey non-discrimination constraints, for instance, by changing the labels of some leaves in a decision tree (Calders and Verwer 2010; Kamiran et al. 2010), or removing selected rules from the set of discovered decision rules (Hajian et al. 2015).”
Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining (2016, methodology): “Discrimination discovery: Measures of discrimination; data analysis techniques for discrimination discovery (multidisciplinary approaches in the economic, legal, and statistical domains); data mining approaches for discrimination discovery including classification rules, K-NN, bayesian networks, pro”
Open the Black Box Data-Driven Explanation of Black Box Decision Systems (2018, methodology): “• Causal extensions of the previous correlation-based measures, e.g., by propensity score reweighing [36] or by probabilistic causation confidence score derived from Suppes-Bayes causal networks [4].”
The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning (2018, background): “…anti-classiﬁcation requires that: Some authors have suggested stronger notions of anti-classiﬁcation that aim to guard against the use of unprotected traits that are proxies for protected attributes (Bonchi et al., 2017; Grgic-Hlaca et al., 2016; Johnson et al., 2016; Qureshi et al., 2016).”