Fairwashing: the risk of rationalization

By Ulrich Matchi Aïvodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, Alain Tapp. 2019

184 citations recorded by the source corpus.

Black-box explanation is the problem of explaining how a machine learning model -- whose internal logic is hidden to the auditor and generally complex -- produces its outcomes. Current approaches for solving this problem include model explanation, outcome explanation as well as model inspection. While these techniques can be beneficial by providing interpretability, they can be used in a negative manner to perform fairwashing, which we define as promoting the false perception that a machine learning model respects some ethical values. In particular, we demonstrate that it is possible to systematically rationalize decisions taken by an unfair black-box model using the model explanation as well as the outcome explanation approaches with a given fairness metric. Our solution, LaundryML, is based on a regularized rule list enumeration algorithm whose objective is to search for fair rule lists approximating an unfair black-box model. We empirically evaluate our rationalization technique on black-box models trained on real-world datasets and show that one can obtain rule lists with high fidelity to the black-box model while being considerably less unfair at the same time.

Subfields: Artificial Intelligence, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In AI Ethics, Fairness & Society

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “The same goal has been recently pursued in [401] , showing that post-hoc XAI techniques can forge fairer explanations from truly unfair black-box models.”
Learning Fair Rule Lists (2019, background): “For instance, local and global explanation techniques can be manipulated to under-report the unfairness of the black-box model (Aı̈vodji et al., 2019).”
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods (2019, background): “[1] demonstrated that post hoc explanations can potentially be exploited to fairwash i.”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, background): “As discussed earlier, the multiplicity of recourse explanations/recommendations (see §4.1.1) may allow for an organization to cherry-pick “the most socially acceptable explanation out of many equally plausible ones” [21, 79, 99] (see also, fairwashing [6]).”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, background): “, “promoting the false perception that a ML model respects some ethical values” [65] is needed in all medical areas where such systems may be utilized.”
Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation (2023, methodology): “This makes it critical to characterize the manipulability of fairwashing [83], as well as LaundryML approaches [82] to better audit unfair opaque models.”
Model-Agnostic Linear Competitors - When Interpretable Models Compete and Collaborate with Black-Box Models (2019, background): “There’s a very recent work that demonstrates that explanations can be deceptive and contrary to the real mechanism in a model (A¨ıvodji et al. 2019).”
Interpretable Companions for Black-Box Models (2020, background): “We emphasize this distinction because black-box explainers serve the purpose of providing post-hoc analysis and approximations to the decision maker, which may not represent the true interactions of features inside the black-box [5, 12].”
The Bouncer Problem: Challenges to Remote Explainability (2019, background): “This problematic is also ref-ered to as rationalization in a paper by Avodji et al [3].”