Explanation in Artificial Intelligence: Insights from the Social Sciences
By Tim Miller. 2017
5,450 citations recorded by the source corpus.
There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to make their algorithms more understandable. Much of this research is focused on explicitly explaining decisions or actions to a human observer, and it should not be controversial to say that looking at how humans explain to each other can serve as a useful starting point for explanation in artificial intelligence. However, it is fair to say that most work in explainable artificial intelligence uses only the researchers' intuition of what constitutes a `good' explanation. There exists vast and valuable bodies of research in philosophy, psychology, and cognitive science of how people define, generate, select, evaluate, and present explanations, which argues that people employ certain cognitive biases and social expectations towards the explanation process. This paper argues that the field of explainable artificial intelligence should build on this existing research, and reviews relevant papers from philosophy, cognitive psychology/science, and social psychology, which study these topics. It draws out some important findings, and discusses ways that these can be infused with work on explainable artificial intelligence.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Social Sciences & Humanities
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Percy Liang
- Noah A. Smith
- Luke S. Zettlemoyer
- Jianfeng Gao
- Yoshua Bengio
- Yoav Goldberg
In Safety, Robustness & Interpretability
- Longo Luca
- Giulia Vilone
- Q. Liao
- Ana Marasovic
- Adrian Weller
- Sarah Wiegreffe
- Wojciech Samek
- K. Müller
In Social Sciences & Humanities
- Kacper Sokol
- Peter A. Flach
- Daniel S. Weld
- Ana Marasovic
- Q. Liao
- Bernhard Schölkopf
- Paul Rad
- Arun Das
Citations in the corpus
- Ethical and social risks of harm from Language Models (2021, background): “…is even some hope that LMs may be useful for improving explainability in other types of AI systems, for example by helping to generate explanations that are accessible and somewhat personalised to a person’s level of knowledge (for an elaboration of such types of explanations see (Miller, 2018)).”
- Understanding Dataset Difficulty with V-Usable Information (2021, positive): “Comparisons of V -usable information before and after isolating an input attribute shed light on why the dataset is easy or difficult for V (§4), which has significant implications for interpretability in AI 2 (Miller, 2019).”
- STaR: Bootstrapping Reasoning With Reasoning (2022, background): “Still, in real-world contexts there are significant consequences to user trust and risk from a model producing bad or unfaithful explanations [36], discussed also in Appendix 6.”
- Measuring Association Between Labels and Free-Text Rationales (2020, background): “In addition to low correlation with human scores, there can be many valid rationales for a given instance (Miller, 2019); metrics that compare generated rationales to a single ground-truth do not address this and are thus a poor measure of quality.”
- Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021, methodology): “Counterfactual reasoning — mentally simulating what would have happened if conditions were different — is a common tool for making causality assessments (Kahneman and Tversky, 1981), which in turn are crucial for model evaluation, error analysis, and explanation (Miller, 2019).”
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Relevant findings to be considered when creating an explainable AI model are highlighted in [12] : First, explanations are better when constrictive , meaning that a prerequisite for a good explanation is that it does not only indicate why the model made a decision X, but also why it made decision X…”
- Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning (2019, background): “It enables the user to consider contrastive explanations and counterfactual analyses, such as why one decision was made instead of another, and to predict how a change to a feature will affect the system’s output [Miller, 2019].”
- Explaining Predictions from Tree-based Boosting Ensembles (2019, background): “(6) Since this ε-perturbation allows us to discriminate between the two classes so it can be viewed as the contrastive explanation [7] for why f̂ (x) = −1 as opposed to +1.”
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, methodology): “…2012, 2011, 2013, 2015; Nataksu, 2004; Ribiero, et al., 2016a,b; Sheh and Monteath, 2008; Treu, 1998; Wickens, 1994), and the challenges for the evaluation of explanation systems (Doshi-Velez and Kim, 2017ab; Gilpin et al., 2018; Feldmann, 2018; Miller, 2017; Miller, Howe, and Sonenberg, 2017).”
- Metrics for Explainable AI: Challenges and Prospects (2018, methodology): “To evaluate this model of the explanation process, a number of types of measures are required (Miller, 2017).”
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “(iii) Human science:to produce artificial explanations, it is worth first to models how humans explain decisions and behavior to each other [33].”
- Explainable Recommendation: A Survey and New Perspectives (2018, background): “not be what we need in many cases (Miller, 2019).”
- Mathematical optimization in classification and regression trees (2021, neutral): “(2018), Miller (2019).”
- Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI (2020, background): “There are many different kinds of explanation that an AI system could supply, so future research on the impact of different kinds of explanation on trust calibration should be guided by knowledge gained in the social sciences on how humans understand explanations.(72,73) Providing uncertainty estima”
- ViCE: visual counterfactual explanations for machine learning models (2020, background): “time interpretable (minimal set of features) and feasible (minimal amount of change); characteristics that are crucial for user-friendly explanations [13].”
- Explainable machine learning in deployment (2019, background): “Though non-causal explanations can still provide valid and useful interpretations of how the model works [37], many organizations said that theywould be keen to use causal explanations”
- Handling Model Uncertainty and Multiplicity in Explanations via Model Reconciliation (2018, methodology): “Explanations of this form have been inspired by works like (Lombrozo 2006; 2012; Miller 2017) which identify properties of explanations in terms of selectivity, contrastiveness and mental modeling of the explainee, and our recent work (Chakraborti et al.”