Explaining machine learning classifiers through diverse counterfactual explanations

By Ramaravind Kommiya Mothilal, Amit Sharma, Chenhao Tan. 2019

1,400 citations recorded by the source corpus.

Post-hoc explanations of machine learning models are crucial for people to understand and act on algorithmic predictions. An intriguing class of explanations is through counterfactuals, hypothetical examples that show people how to obtain a different prediction. We posit that effective counterfactual explanations should satisfy two properties: feasibility of the counterfactual actions given user context and constraints, and diversity among the counterfactuals presented. To this end, we propose a framework for generating and evaluating a diverse set of counterfactual explanations based on determinantal point processes. To evaluate the actionability of counterfactuals, we provide metrics that enable comparison of counterfactual-based methods to other local explanation methods. We further address necessary tradeoffs and point to causal implications in optimizing for counterfactuals. Our experiments on four real-world datasets show that our framework can generate a set of counterfactuals that are diverse and well approximate local decision boundaries, outperforming prior approaches to generating diverse counterfactuals. We provide an implementation of the framework at https://github.com/microsoft/DiCE.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

On the Fairness of Causal Algorithmic Recourse (2020, background): “…to change their situation may have downstream effects (Mahajan, Tan, and Sharma 2019; Karimi, Schölkopf, and Valera 2021; Karimi et al. 2020c; Mothilal, Sharma, and Tan 2020), cf. also (Barocas, Selbst, and Raghavan 2020; Wachter, Mittelstadt, and Russell 2017; Ustun, Spangher, and Liu…”
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, background): “…x CE for an individual x F is given by a solution to the following optimisation problem: where dist ( · , · ) is a similarity metric on X , and additional constraints may be added to reﬂect plausibility, feasibility, or diversity of the obtained counterfactual explanations [15, 16, 24, 25, 31, 35].”
ViCE: visual counterfactual explanations for machine learning models (2020, background): “It is also worth noticing that the visualization interface can accommodate any other counterfactual generation methods [9, 16, 21, 22].”
Interpretable Counterfactual Explanations Guided by Prototypes (2019, background): “Some previously proposed solutions are either computationally expensive [27] or do not take relationships between categories into account [9, 22].”
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models (2020, background): “And some provide rich support for counterfactual analysis, either within-dataset (What-If Tool) or dynamically generated as in DiCE (Mothilal et al., 2020).”
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2021, background): “A complementary approach is to generate counterfactuals with minimal changes that obtain a different model prediction (Wachter et al., 2017; Mothilal et al., 2020).”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences (2020, methodology): “We conducted semi-structured interviews with 20 UX and de-sign practitioners recruited from multiple product lines at IBM.”
A historical perspective of explainable Artificial Intelligence (2020, methodology): “…the classifier's decision boundary (Ribeiro et al., 2016a), counterfactual explanations have the advantage that they are more human understandable (Mothilal et al., 2020), and that they are always truthful with respect to the underlying model by giving direct outputs of the algorithm (Wachter,…”
Explainable AI (XAI): Core Ideas, Techniques, and Solutions (2022, background): “DiCE [45] is a package that gives the “what-if" explanations for the model prediction.”
Benchmarking and survey of explanation methods for black box models (2021, background): “…et al. (2019) ANY IN G A tsp Tan et al. (2020) TAB PH L S CF cem Dhurandhar et al. (2018) ANY PH L S cfx Albini et al. (2020) TAB PH L S dice Mothilal et al. (2020) TAB PH L A c- chave Pawelczyk et al. (2020) TAB PH L A face Poyiadzi et al. (2020) ANY PH L A Ares Ley et al. (2022) TAB PH G…”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “We hope the taxonomy can shed light on future improvements/extensions on explaining Table 2 List of interpretation algorithm publications Methods Publications (non-exhaustive) LIME and variants LIME [137], Anchors [138], SHAP [110], RISE [127], MAPLE [130] Global interpretation LIME-SP [137], NormLI”
Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification (2021, positive): “The tool of causality analysis has been successfully used in several areas, including explainable machine learning [40], fairness [24], natural language processing [61], reinforcement learning [20] and adversarial learning [22].”
The Use and Misuse of Counterfactuals in Ethical Machine Learning (2021, background): “The use of counterfactuals has become increasingly popular in the machine learning community for many reasons such as making sense of algorithmic fairness or explainability in automated decision-making for consequential social contexts [4, 9, 14, 18, 29, 35, 42, 47, 51].”
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, background): “independent features is evaluated [105].”
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences (2021, methodology): “Example-based methods can also be used to generate counterfactual examples–instances with minimum differences from the original one but having a different outcome [76, 97].”