LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

By Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, Alexander M. Rush. 2016

441 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Human-Computer Interaction, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Human-Computer Interaction

In Signal Processing

Citations in the corpus

HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, neutral): “We highlight specific systems developed by users with different goals following the architect, trainer, and end-user distinction of Strobelt et al. (2017):”
HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, positive): “We highlight specific systems developed by users with different goals following the architect, trainer, and end-user distinction of Strobelt et al. (2017):”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “Seq2Seq-Vis (Strobelt et al., 2018a) visualizes different modules in attention-based seq2seq models, with the goal of examining model decisions and testing alternative decisions.”
Interpretable and Steerable Sequence Learning via Prototypes (2019, background): “) sequence models with built-in interpretability. Post-hoc explanation. Post-hoc methods unveil the underlying mechanisms of a pre-existing “black-box” model. Karpathy et al. [17] and Strobelt et al. [31] visualize the changes of the internal states in RNNs to understand the roles of the hidden unit”
An Analysis of Encoder Representations in Transformer-Based Machine Translation (2018, background): “Traditionally, a common way to inspect neural networks is by visualizing the hidden representation trained for a specific task (Ding et al., 2017; Strobelt et al., 2018a,b), and to evaluate them by assessing the properties through downstream tasks (Chung et al., 2014; Greff et al., 2017).”
Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs (2018, background): “Another line of work (Karpathy et al., 2015) (Strobelt et al., 2016) has focused on analysing the movement of raw gate activations over a sequence.”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers (2018, unknown): “It is encouraging to see many of the visual analytics systems recognize this importance and report on design studies conducted with AI experts before building a tool to understand the users and their needs [14], [15], [39], [43], [52].”
Interpreting Word-Level Hidden State Behaviour of Character-Level LSTM Language Models (2018, methodology): “More recently, Li et al. (2016) directly visualize representations over time using heatmaps, and Strobelt et al. (2018) develop interactive tools for visualizing LSTM hidden states and testing hypotheses about distributed representations.”
Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models (2018, background): “As discussed earlier, many existing visual analytics solutions focus on interpreting the internal working mechanism of a specific model type [12, 21, 25, 32, 37, 41].”
Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “For instance, heat-maps can highlight specific areas of an image or specific words of a text that mostly influence the inferential process of a model by using different colours [134, 135].”
Transformers: State-of-the-Art Natural Language Processing (2020, unknown): “We highlight specific systems developed by users with different goals following the architect, trainer, and end-user distinction of Strobelt et al. (2017):”
RuleMatrix: Visualizing and Understanding Classifiers with Rules (2018, methodology): “More recently, a number of visual analytics methods have been developed to support the analysis of complex deep neural networks [5, 20, 21, 25, 27, 30, 35, 40].”
Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules? (2018, unknown): “Hence, current interpretations in NLP are limited to visualizing the magnitude of the LSTM hidden states of each word (Linzen et al., 2016; Radford et al., 2017; Strobelt et al., 2018), removing words (Li et al.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Attention and gating are frequently used in NLP models, and there have been plenty of works aiming to understand the model through these weights, such as Rollout [2], Seq2Seq-Vis [157] and others [61, 158], or to investigate the reliability of using them as explanations [82, 148, 177].”