Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems

By Shalmali Joshi, Oluwasanmi Koyejo, Warut D. Vijitbenjaronk, Been Kim, Joydeep Ghosh. 2019

225 citations recorded by the source corpus.

Machine learning based decision making systems are increasingly affecting humans. An individual can suffer an undesirable outcome under such decision making systems (e.g. denied credit) irrespective of whether the decision is fair or accurate. Individual recourse pertains to the problem of providing an actionable set of changes a person can undertake in order to improve their outcome. We propose a recourse algorithm that models the underlying data distribution or manifold. We then provide a mechanism to generate the smallest set of changes that will improve an individual's outcome. This mechanism can be easily used to provide recourse for any differentiable machine learning based decision making system. Further, the resulting algorithm is shown to be applicable to both supervised classification and causal decision making systems. Our work attempts to fill gaps in existing fairness literature that have primarily focused on discovering and/or algorithmically enforcing fairness constraints on decision making systems. This work also provides an alternative approach to generating counterfactual explanations.

Subfields: Artificial Intelligence, Information Systems and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems and Management

Citations in the corpus

On the Fairness of Causal Algorithmic Recourse (2020, background): “…with offering recommendations to individuals, who were unfavourably treated by a decision-making system, to overcome their adverse situation (Joshi et al. 2019; Ustun, Spangher, and Liu 2019; Sharma, Henderson, and Ghosh 2019; Mahajan, Tan, and Sharma 2019; Mothilal, Sharma, and Tan 2020;…”
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, background): “5, (2) where dist(·, ·) is a similarity metric on X , and additional constraints may be added to reflect plausibility, feasibility, or diversity of the obtained counterfactual explanations [19, 20, 32, 33, 39, 44].”
Deep Neural Networks and Tabular Data: A Survey (2021, background): “To alleviate this problem and to suggest realistic-looking counterfactuals, researchers have suggested building recourse suggestions on generative models [178], [179], [180], [181], [182].”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, background): “2 Related Work Most post-hoc interpretability methods fall under the categories: (a) feature-based explanation methods, that attribute the decision to important input features [1, 2, 13, 14], (b) sample-based explanation methods, that attribute the decision to previously observed samples [15, 16, 17”
Algorithmic Recourse: from Counterfactual Explanations to Interventions (2020, background): “…x CFE ∗ ∈ X is a (perhaps not unique) nearest counterfactual instance; h θ is the ﬁxed predictor; and P is an optional set of plausibility constraints, e.g., the counterfactual instance be from a relatively high density region of the input space (Joshi et al., 2019; Poyiadzi et al., 2019).”
Counterfactual Explanations for Machine Learning: A Review (2020, methodology): “[61] Gradients Differentiable No No No Yes No No [90] Gradients Differentiable No No No No No No [113] Blackbox Agnostic No No Changes one feature No No No -”
The Use and Misuse of Counterfactuals in Ethical Machine Learning (2021, background): “In machine learning practice, there are several technical ways to generate and evaluate counterfactuals, such as featurebased explanations, prototype explanations, example-based explanations, or causal explanations [19, 32, 35, 38, 46, 50, 51].”
The philosophical basis of algorithmic recourse (2020, background): “Indeed, recent articles seeks to do precisely this via estimating the submanifold via data sampling[17].”
FOCUS: Flexible Optimizable Counterfactual Explanations for Tree Ensembles (2019, background): “Algorithmic recourse is a line of research that is closely related to counterfactual explanations, except that these methods include the additional restriction that the resulting explanation must be actionable [16, 19, 20, 34].”
Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers (2019, methodology): “Most CF generation methods rely on separate optimizations for each input, based on original features [16, 7], a latent representation [17], or using a generative adversarial network [18].”
Multi-Objective Counterfactual Explanations (2020, unknown): “te predictions. A model-specic approach was proposed by Wachter et al. [35], who also introduced and formalized the concept of counterfactuals in predictive modeling. Like many model-specic methods [15,20,24,28,33] their approach is limited to dierentiable models. The approach of Tolomei et al. [32]”