Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers

By Fred Hohman, Minsuk Kahng, Robert S. Pienta, Duen Horng Chau. 2018

606 citations recorded by the source corpus.

Deep learning has recently seen rapid development and received significant attention due to its state-of-the-art performance on previously-thought hard problems. However, because of the internal complexity and nonlinear structure of deep neural networks, the underlying decision making processes for why these models are achieving such performance are challenging and sometimes mystifying to interpret. As deep learning spreads across domains, it is of paramount importance that we equip users of deep learning with tools for understanding when a model works correctly, when it fails, and ultimately how to improve its performance. Standardized toolkits for building neural networks have helped democratize deep learning; visual analytics systems have now been developed to support model explanation, interpretation, debugging, and improvement. We present a survey of the role of visual analytics in deep learning research, which highlights its short yet impactful history and thoroughly summarizes the state-of-the-art using a human-centered interrogative framework, focusing on the Five W's and How (Why, Who, What, How, When, and Where). We conclude by highlighting research directions and open research problems. This survey helps researchers and practitioners in both visual analytics and deep learning to quickly learn key aspects of this young and rapidly growing body of research, whose impact spans a diverse range of domains.

Subfields: Biophysics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “[159] where a survey of the role of visual analytics in deep learning is presented, the authors acknowledge the importance of producing visualizations and interpretations for DNN that are human understandable and expose”
Deep learning-based electroencephalography analysis: a systematic review (2019, methodology): “For a general review of DL model inspection techniques, see [84].”
The Role of Cooperation in Responsible AI Development (2019, background): “We still do not fully understand the decisions made by complex machine learning models (Olah et al., 2018; Hohman et al., 2018) and the high-dimensionality of the inputs to AI systems makes it such that exhaustive enumeration of all possible inputs and outputs is typically infeasible.”
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization (2020, background): “In recent years, many visualization systems have been developed for deep learning, but very few are designed for non-experts [20, 29, 44, 50], as surveyed in [23].”
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (2019, methodology): “Since then, interactive visual analytics has succeeded in supporting machine learning tasks [25, 41, 42, 55, 63].”
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence (2019, background): “Practically, end users are less likely to trust and cede control to machines whose workings they do not understand (Burrell 2016; Ribeiro et al. 2016), and software developers may be unable to intervene in order to quickly and systematically improve performance (Hohman et al. 2018).”
Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models (2018, neutral): “[13], our tool aims at model developers who have at least a conceptual understanding of how the model works.”
Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities (2020, methodology): “For instance, AI Developers focus on improving the algorithm’s performance as well as on debugging and verification in order to pursue a structured engineering approach based on cause analysis instead of trial and error (Hohman et al., 2019).”
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders (2019, background): “Two recent review papers [1, 26] both suggest bridging these two isolated research areas, by drawing principles and methods from HCI to improve the usability of explanation interfaces and performing empirical studies to evaluate the efficacy of these interfaces.”
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference (2019, methodology): “These methods also commonly assist in a pedagogical context [33,69].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “f explanation has also been implemented in recommendation systems[8,44] and robotics [104] Analytic explanation is another approach to see and explore data and machine learning models representations [49] Analytic explanations commonly rely on numerical metrics and data visualizations. For instance,”
The What-If Tool: Interactive Probing of Machine Learning Models (2019, background): “The challenge of understanding ML performance has inspired many systems (see [12] for a sample survey, or [24] for a general analysis of the space).”
Deep-learning seismology (2022, methodology): “…how a deep-learning model works, gaining useful information about learned models is possible by using interpretation tools such as backward propagation techniques, saliency maps, or heatmaps to reveal relevant patterns in the input based on feature importance or relevance scores ( 143 , 144 ).”
Semantic bottleneck for computer vision tasks (2018, background): “(explanation, justiﬁcation, transparency, etc. ) Several recent papers have tried to clarify those expressions [20,3,21,4,22,2,23,5] and separate the various approaches in two goals: build interpretable models and/or provide justiﬁ-cation of the prediction.”