Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning

By Ruth M. J. Byrne. 2019

339 citations recorded by the source corpus.

Counterfactuals about what could have happened are increasingly used in an array of Artificial Intelligence (AI) applications, and especially in explainable AI (XAI). Counterfactuals can aid the provision of interpretable models to make the decisions of inscrutable systems intelligible to developers and users. However, not all counterfactuals are equally helpful in assisting human comprehension. Discoveries about the nature of the counterfactuals that humans create are a helpful guide to maximize the effectiveness of counterfactual use in AI.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “It was also shown that the use of counterfactual explanations can help the user to understand the decision of a model [40,42,351] .”
Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, background): “As such, counterfactual explanations can be seen as a case of contrastive explanations (Dhurandhar et al., 2018; Byrne, 2019).”
Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning (2019, background): “Humans use counterfactuals to reason about the enabling conditions of particular outcomes, as well as to identify situations where the outcome would have occurred even in the absence of some action or condition (de Graaf & Malle, 2017; Byrne, 2019).”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, background): “For the interested reader, an in-depth discussion on the social, philosophical, cognitive, and psychological aspects of explanations can be found at [21, 35, 125, 174].”
Factual and Counterfactual Explanations for Black Box Decision Making (2019, background): “the cognitive psychology literature maintaining that counterfactuals help people to reason on explanations that identify cause-effect or reason-action relations between events.(5,6) Similarly, Grover et al.”
Benchmarking and survey of explanation methods for black box models (2021, background): “…we highlight that the literature reviews related to explainability are focused not just on ML and AI but also on social studies (Miller 2019; Byrne 2019), recommendation systems (Zhang and Chen 2020), model-agents(Anjomshoae et al. 2019), and domain-specific applications such as health and…”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “One is the short survey [21] from 2019 by Byrne.”
Explainable Artificial Intelligence: a Systematic Review (2020, background): “Counterfactuals add also something new to what is already known from the existing data and provide additional information on how a model behaves in novel, unseen situations [36].”
Counterfactual Explanations for Machine Learning: A Review (2020, background): “Psychologists have demonstrated that counterfactuals elicit causal reasoning in humans [20, 21, 62].”
Explainable Reinforcement Learning Through a Causal Lens (2019, background): “Why not questions let the explainee ask why an event has not occurred, thus allowing counterfactuals to be explained; something that is known to be a powerful explanation mechanism (Miller 2018b; Byrne 2019).”
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, background): “trastive by nature and therefore present a source of valuable complementary information to a given automatic prediction [52].”
Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making (2021, background): “Meanwhile, reviews of the social science literature reveal that humans tend to provide contrastive explanations when explaining their decisions to each other, suggesting that explaining an AI using counterfactual examples can be most understandable to humans as they share similar conceptual framewor”
Explainable artificial intelligence: an analytical review (2021, background): “On one side, neuroscience and psychology can help build rationalized XAI models that are more easily understood by humankind (Byrne, 2019; Taylor & Taylor, 2020).”
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review (2020, neutral): “Even before the emergence of counterfactual explanations in applied fields, they have been the topic of discussion in fields like social sciences [238], philosophy [194, 215, 286], and psychology [49, 50, 178].”