Towards Better Analysis of Deep Convolutional Neural Networks

By Mengchen Liu, Jiaxin Shi, Z. Li, Chongxuan Li, Jun Zhu, Shixia Liu. 2016

514 citations recorded by the source corpus.

Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification. However, the development of high-quality deep models typically relies on a substantial amount of trial-and-error, as there is still no clear understanding of when and why a deep model works. In this paper, we present a visual analytics approach for better understanding, diagnosing, and refining deep CNNs. We formulate a deep CNN as a directed acyclic graph. Based on this formulation, a hybrid visualization is developed to disclose the multiple facets of each neuron and the interactions between them. In particular, we introduce a hierarchical rectangle packing algorithm and a matrix reordering algorithm to show the derived features of a neuron cluster. We also propose a biclustering-based edge bundling method to reduce visual clutter caused by a large number of connections between neurons. We evaluated our method on a set of CNNs and the results are generally favorable.

Subfields: Biophysics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Improving Interpretability of Deep Neural Networks with Semantic Information (2017, background): “Although previous works [37, 23] have provided some applications on human interaction and architecture selection, they still need expert-level users to join the procedure because non-expert users get little insight about potential problems.”
Opportunities and obstacles for deep learning in biology and medicine (2017, background): “ActiVis and CNNvis [362,363] are two frameworks that enable interactive visualization and exploration of large-scale deep learning models.”
DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation (2019, background): “The model-based solutions focus on revealing the internal architecture of a DNN and the interaction between different components [19], [20], [21].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, unknown): “CNNs for image classification [233], [234].”
Understanding convolutional neural networks with a mathematical model (2016, background): “Keywords: Convolutional Neural Network (CNN), Nonlinear Activation, RECOS Model, Rectified Linear Unit (ReLU), MNIST Dataset.”
Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations (2020, methodology): “, 2018) have been widely used to visualize the high dimensional activation-space of neural networks to identify the role of network areas in solving a given task (Liu et al., 2016; Rauber et al., 2016; Elloumi et al., 2018; V., 2019).”
Comprehensive Review of Artificial Neural Network Applications to Pattern Recognition (2019, background): “[171] on deep recursions introduced a visual analytics technique to understand further, diagnose, and refine DNN.”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “o intelligent agents and context-aware systems. Further, visualization surveys follow visual analytics goals like understanding and interacting with machine learning systems in different applications [75,108,3,31]. In a recent example, Homan et al. [49] provide an excellent review and categorization”
Do Convolutional Neural Networks Learn Class Hierarchy? (2017, methodology): “CNNVis [39] is a powerful system designed to diagnose deep CNNs.”
DeepClue: Visual Interpretation of Text-Based Deep Stock Prediction (2019, methodology): “CNNVis introduced several novel visualization algorithms such as hierarchical rectangle packing and matrix reordering to display features on clustered neurons.”
Towards better analysis of machine learning models: A visual analytics perspective (2017, methodology): “More recently, there have been some initial efforts on diagnosing deep learning models [2, 18].”
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization (2020, background): “Many visual analytics tools have been developed to help deep learning experts analyze their models and predictions [5,15,23,27,36,37].”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers (2018, unknown): “Regarding neural network architecture, many visual analytics systems use node-link diagrams where neurons are nodes and weights are links [13], [14], [16], [35], [37].”
Interpreting Black-Box Classifiers Using Instance-Level Visual Explanations (2017, methodology): “For instance, logistic regression is often used to create a transparent weighting of the features and visualization systems have been developed to visualize decisions trees [22] and neural networks [14, 19].”
RuleMatrix: Visualizing and Understanding Classifiers with Rules (2018, methodology): “More recently, a number of visual analytics methods have been developed to support the analysis of complex deep neural networks [5, 20, 21, 25, 27, 30, 35, 40].”
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks (2016, methodology): “[20] utilize a DAG metaphor to show neurons, their connections, and learned features.”
Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow (2018, background): “Some projects present visualizations for specific types of neural networks such as convolutional network [39] and recurrent networks [2, 48].”