"I know it when I see it". Visualization and Intuitive Interpretability
By Fabian Offert. 2017
10 citations recorded by the source corpus.
Most research on the interpretability of machine learning systems focuses on the development of a more rigorous notion of interpretability. I suggest that a better understanding of the deficiencies of the intuitive notion of interpretability is needed as well. I show that visualization enables but also impedes intuitive interpretability, as it presupposes two levels of technical pre-interpretation: dimensionality reduction and regularization. Furthermore, I argue that the use of positive concepts to emulate the distributed semantic structure of machine learning models introduces a significant human bias into the model. As a consequence, I suggest that, if intuitive interpretability is needed, singular representations of internal model states should be avoided.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohammed Berrada
- Amina Adadi
- Longo Luca
- Giulia Vilone
- Gesina Schwalbe
- Bettina Finzel
- Christian Walter Peter Omlin
- Waddah Saeed
Citations in the corpus
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “In contrast to this wave of thoughts, Offert [163] suggested in his work ‘‘I know it when I see it’’ that a better understanding of the deficiencies of the intuitive notion of interpretability is needed as well.”
- Explainable Artificial Intelligence: a Systematic Review (2020, unknown): “The explanandum is something unexpected by the user that creates dissonance with his/her beliefs [58] Correctability The capacity of a method for explainability to allow end-users make technical adjustments to an underlying model [60, 61] Simplification The capacity to reduce the number of variables”