The misunderstood limits of folk science: an illusion of explanatory depth

By Leon Rozenblit, Frank C. Keil. 2002

892 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, General Decision Sciences, Cognitive Science & Psychology, Philosophy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In General Decision Sciences

In Cognitive Science & Psychology

In Philosophy

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, background): “…explanation, especially depth and detail that can be distracting (Goetz, and Sadoski, 1995), or the "illusion of explanatory depth" in which people feel they sufficiently understand something, and overestimate how well they understand (e.g., Rozenblit and Keil, 2002; Van Der Linden et al., 2017).”
Objecting to experiments that compare two unobjectionable policies or treatments (2019, neutral): “, the FDA) either already do or should know what the correct treatment is, a proxy form of illusion of knowledge (20).”
Explanation and Abductive Inference (2012, background): “…human cognition, the complete explanation could be quite incomplete—it's likely that human explanatory practice proceeds in the absence of fully specified explanations, even if there is always a more complete explanation that underlies the selected explanation (Keil, 2003; Rozenblit & Keil, 2002).”
The Instrumental Value of Explanations (2011, background): “Rozenblit and Keil (2002), for example, document a phenomenon they call the Illusion of Explanatory Depth.”
Explanations in the wild. (2021, positive): “Work on the ‘Illusion of Explanatory Depth’ (Rozenblit & Keil, 2002) has shown that people overestimate their own understanding before explaining something technical, only to realize the limits of their knowledge once they attempt to generate an explanation.”
Evolutionary neuromarketing: darwinizing the neuroimaging paradigm for consumer behavior (2008, background): “It is important to reiterate that despite the richness of scientific fMRI-based empirical work, one might argue that the paradigm suffers from an illusion of explanatory depth (Rozenblit and Keil, 2002).”
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI (2021, methodology): “In this paper, we argue that because of this illusion of explanatory depth (IOED) [46], XAI explanations (especially in the form of additive local explanations for individual predictions) may be misleading for non-technical XAI users.”
AXIS: Generating Explanations at Scale with Learnersourcing and Machine Learning (2016, background): “However, an extensive literature reveals learners’ failures in metacognitive awareness of what they do and do not know, such as the illusion of explanatory depth [22] reveal people’s great surprise at their erroneous assumptions about being able to provide detailed explanations.”
Direct-fit to nature: an evolutionary perspective on biological (and artificial) neural networks (2019, background): “These experimental manipulations are often inspired by our ‘‘folk’’ or phenomenological understanding of the mind, brain, or world and, in turn, yield results reﬂecting our own assumptions (Meehl, 1990; Rozenblit and Keil, 2002; Jolly and Chang, 2019 ).”
What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research (2021, background): “Translated to the case of artificial systems, this means that we could examine understanding by investigating stakeholders’ mental models of a system [17, 108]: how well does a person’s mental model mirror the actual system? Are there gaps in the current understanding of a system’s functioning? Are ”