Model-Agnostic Counterfactual Explanations for Consequential Decisions

By Amir-Hossein Karimi, Gilles Barthe, Borja Balle, Isabel Valera. 2019

405 citations recorded by the source corpus.

Predictive models are being increasingly used to support consequential decision making at the individual level in contexts such as pretrial bail and loan approval. As a result, there is increasing social and legal pressure to provide explanations that help the affected individuals not only to understand why a prediction was output, but also how to act to obtain a desired outcome. To this end, several works have proposed optimization-based methods to generate nearest counterfactual explanations. However, these methods are often restricted to a particular subset of models (e.g., decision trees or linear models) and differentiable distance functions. In contrast, we build on standard theory and tools from formal verification and propose a novel algorithm that solves a sequence of satisfiability problems, where both the distance function (objective) and predictive model (constraints) are represented as logic formulae. As shown by our experiments on real-world data, our algorithm is: i) model-agnostic ({non-}linear, {non-}differentiable, {non-}convex); ii) data-type-agnostic (heterogeneous features); iii) distance-agnostic ($\ell_0, \ell_1, \ell_\infty$, and combinations thereof); iv) able to generate plausible and diverse counterfactuals for any sample (i.e., 100% coverage); and v) at provably optimal distances.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

On the Fairness of Causal Algorithmic Recourse (2020, methodology): “We extend the probabilistic framework of Karimi et al. (2020c) to consider causal recourse in the presence of heterogeneous features, see Appendix B.2 for more details.”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, methodology): “Coverage is defined as the number of individuals for which the algorithm can identify a plausible counterfactual instance (through either recourse type), if at least one solution existed [91].”
Multi-Objective Counterfactual Explanations (2020, methodology): “Several model-agnostic counterfactual methods have been proposed [16,9,6,14,23,27,33].”
Benchmarking and survey of explanation methods for black box models (2021, background): “Examples are validity, minimality, similarity, plausibility, discriminative power, actionability, causality, diversity, efficiency, robustness, etc. (Wachter et al. 2017; Karimi et al. 2020a; Kanamori et al. 2020).”
Deep Neural Networks and Tabular Data: A Survey (2021, methodology): “In the class of independence-based methods, where the input features of the predictive model are assumed to be independent, some approaches use combinatorial solvers to generate recourse in the presence of feasibility constraints [172], [173], [174], [175].”
Counterfactual Explanations for Machine Learning: A Review (2020, methodology): “As examples, gradient based algorithms can only handle differentiable models, the algorithms based on solvers require linear or piece-wise linear models [95, 107, 64, 65, 63], some algorithms are model-speciﬁc and only work for those models like tree ensembles [104, 63, 47, 79].”
Algorithmic Recourse: from Counterfactual Explanations to Interventions (2020, methodology): “, 2019), or generated as in (Wachter et al., 2017; Ustun et al., 2019; Karimi et al., 2019) among others.”
Explaining NLP Models via Minimal Contrastive Editing (MiCE) (2020, background): “The problem of generating minimal contrastive edits, also called counterfactual explanations (Wachter et al., 2017), 12 has previously been explored for tabular data (Karimi et al., 2020) and images (Hendricks et al., 2018; Goyal et al., 2019; Looveren and Klaise, 2019) but less for language.”
The hidden assumptions behind counterfactual explanations and principal reasons (2019, background): “differently from the point currently in question [20, 29, 36, 44, 47].”
FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles (2019, methodology): “Fidelity is commonly used to evaluate XAI methods that are based on approximations of a givenmodel: it is ameasure of the agreement between the original modelM and its approximation M̃: fid(M̃, 𝑋 )= 1 𝑁 𝑁∑︁ 𝑛=1 1 [ 𝑦𝑥 (𝑛) =arg max 𝑦′ M̃(𝑦′ |𝑥 (𝑛) ) ] , (26) where 𝑦𝑥 = arg max𝑦 M(𝑦 | �”