ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

By Minsuk Kahng, Pierre Yves Andrews, Aditya Kalro, Duen Horng Chau. 2017

351 citations recorded by the source corpus.

While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge. Despite the recent interest in developing visual tools to help users interpret deep learning models, the complexity and wide variety of models deployed in industry, and the large-scale datasets that they used, pose unique design challenges that are inadequately addressed by existing work. Through participatory design sessions with over 15 researchers and engineers at Facebook, we have developed, deployed, and iteratively improved ActiVis, an interactive visualization system for interpreting large-scale deep learning models and results. By tightly integrating multiple coordinated views, such as a computation graph overview of the model architecture, and a neuron activation view for pattern discovery and comparison, users can explore complex deep neural network models at both the instance-and subset-level. ActiVis has been deployed on Facebook's machine learning platform. We present case studies with Facebook researchers and engineers, and usage scenarios of how ActiVis may work with different models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Opportunities and obstacles for deep learning in biology and medicine (2017, background): “ActiVis and CNNvis [362,363] are two frameworks that enable interactive visualization and exploration of large-scale deep learning models.”
Errudite: Scalable, Reproducible, and Testable Error Analysis (2019, background): “ActiVis (Kahng et al., 2018) allows for ﬂexible data attribute and group deﬁnitions, but only supports group creation prior to the interactive process.”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers (2018, neutral): “ActiVis [39]: a visual analytics system for interpreting neural network results using a novel visualization that unifies instance- and subset-level inspections of neuron activations deployed at Facebook.”
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization (2020, neutral): “hine learning algorithms. 3.3 Visual Analytics for Neural Networks & Predictions Many visual analytics tools have been developed to help deep learning experts analyze their models and predictions [5,15,23,27,36,37]. These tools support many different tasks. For example, recent work such as Summi”
DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation (2019, background): “Conversely, the example-based solutions focus on interpreting the behavior of DNNs with specific data instances as inputs [22], [23].”
Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models (2018, neutral): “While the current state-of-the art for diagnosing and improving deep neural network models is still in an early stage [16, 17, 28, 47], our goal is to allow the user to test specific interventions.”
RuleMatrix: Visualizing and Understanding Classifiers with Rules (2018, background): “[16] developed ActiVis, a visual system to support the exploration of industrial deep learning models in Facebook.”
Designing Theory-Driven User-Centric Explainable AI (2019, background): “Leveraging on inductive reasoning, scatterplot diagrams help users to perceive the similarity between objects by presenting objects in lower dimensionality projection (e.g., t-SNE) [46].”
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (2019, methodology): “Example tasks include interactive model debugging and performance analysis [2, 44, 53], feature ideation and selection [6, 34], instance subset inspection and comparison [30, 31], model comparison [67], and constructing interpretable and trustworthy visual analysis systems [11].”
Biased Embeddings from Wild Data: Measuring, Understanding and Removing (2018, background): “…growing concern about potential biases in learning systems [1], [6] which can be difﬁcult to analyse or query for explanations of their predictions, leading to an increasing number of studies investigating the way black-box systems represent knowledge and make decisions [7], [9], [11], [19], [20].”
Machine Learning Interpretability: A Survey on Methods and Metrics (2019, methodology): “Facebook, in collaboration with Georgia Tech, published a paper where it shows a visual exploration tool of industry-scale DNN models [39].”
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference (2019, background): “…[20], Rationals [54], Saliency [55], Structured Interpretation [22], Prediction Difference [92] Interactive Observation [Understanding] ActiVis [41], Deep Visualization [89], Embed-ding Projector [70], GANViz [79], LSTMVis [74], Prospector [48], RetainVis [51], RNNVis [62], ShapeShop [32]…”
Explainable Neural Networks based on Additive Index Models (2018, methodology): “There are also eﬀorts to understand neural networks using visualization–based techniques such as those described in Kahng et al. (2017) or Olah et al. (2017).”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “2018 [95] ◆ ◆ ◆”
Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models (2018, background): “Typical solutions include visualizing the internal structure or intermediate states of the model to enhance the understanding and interpretation [12,25,41], evaluating and analyzing the performance of models or algorithms [5, 22, 33], and interactively improving the models at different development s”