Understanding Hidden Memories of Recurrent Neural Networks

By Yao Ming, Shaozu Cao, Ruixiang Zhang, Z. Li, Yuanzhe Chen, Yangqiu Song, Huamin Qu. 2017

223 citations recorded by the source corpus.

Recurrent neural networks (RNNs) have been successfully applied to various natural language processing (NLP) tasks and achieved better results than conventional methods. However, the lack of understanding of the mechanisms behind their effectiveness limits further improvements on their architectures. In this paper, we present a visual analytics method for understanding and comparing RNN models for NLP tasks. We propose a technique to explain the function of individual hidden state units based on their expected response to input texts. We then co-cluster hidden state units and words based on the expected response and visualize co-clustering results as memory chips and word clouds to provide more structured knowledge on RNNs’ hidden states. We also propose a glyph-based sequence visualization based on aggregate information to analyze the behavior of an RNN’s hidden state at the sentence-level. The usability and effectiveness of our method are demonstrated through case studies and reviews from domain experts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Model Architectures

Citations in the corpus

Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models (2018, unknown): “[32] propose RNNVis, a tool that uses word clouds instead of full contexts or sentences to show typical words that appear for activation patterns.”
LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation (2018, methodology): “Ming et al. (2017) proposed a tool, RNNVis to visualize hidden states based on RNN’s expected response to a r X i v : 1808 .”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “2017 [143] ◆ ◆ ◆”
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use (2019, methodology): “Visualization is also explored as a method of providing an insight in model behaviour (Yosinski et al., 2015; Ming et al., 2017).”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers (2018, unknown): “Similarly to comparing model architectures, some systems solely rely data visualization representations and encodings in order to compare models [43], while others compare different snapshots of a single model as it trains over time, i.”
Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models (2018, background): “Typical solutions include visualizing the internal structure or intermediate states of the model to enhance the understanding and interpretation [12,25,41], evaluating and analyzing the performance of models or algorithms [5, 22, 33], and interactively improving the models at different development s”
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference (2019, background): “…ActiVis [41], Deep Visualization [89], Embed-ding Projector [70], GANViz [79], LSTMVis [74], Prospector [48], RetainVis [51], RNNVis [62], ShapeShop [32] Instance-Level Explanations [47], Manifold [91], NLIZE [59], RNNBow [14], Semantic Image Synthesis [16], Seq2Seq-Vis [73]…”
RuleMatrix: Visualizing and Understanding Classifiers with Rules (2018, methodology): “More recently, a number of visual analytics methods have been developed to support the analysis of complex deep neural networks [5, 20, 21, 25, 27, 30, 35, 40].”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, background): “The visualization of RNNs is more complicated than CNNs as an input may arouse response from every neuron, and a hidden-state neuron may be highly responsive to a set of words, forming many-to-many relationships [Ming et al., 2017].”