Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models

By Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander M. Rush. 2018

262 citations recorded by the source corpus.

Neural sequence-to-sequence models have proven to be accurate and robust for many sequence prediction tasks, and have become the standard approach for automatic translation of text. The models work with a five-stage blackbox pipeline that begins with encoding a source sequence to a vector space and then decoding out to a new target sequence. This process is now standard, but like many deep learning methods remains quite difficult to understand or debug. In this work, we present a visual analysis tool that allows interaction and “what if”-style exploration of trained sequence-to-sequence models through each stage of the translation process. The aim is to identify which patterns have been learned, to detect model errors, and to probe the model with counterfactual scenario. We demonstrate the utility of our tool through several real-world sequence-to-sequence use cases on large-scale models.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Multiscale Visualization of Attention in the Transformer Model (2019, methodology): “Finally, we would like to enable users to manipulate the model, either by modifying attention (Lee et al., 2017; Liu et al., 2018; Strobelt et al., 2018) or editing individual neurons (Bau et al., 2019).”
An Analysis of Encoder Representations in Transformer-Based Machine Translation (2018, background): “Traditionally, a common way to inspect neural networks is by visualizing the hidden representation trained for a specific task (Ding et al., 2017; Strobelt et al., 2018a,b), and to evaluate them by assessing the properties through downstream tasks (Chung et al., 2014; Greff et al., 2017).”
AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models (2019, background): “For instance, Liu et al. (2018), Strobelt et al. (2019), and Vig (2019) visualize attention weights for specific NLP models, while Lee et al. (2019) apply adversarial attacks to reading comprehension systems.”
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models (2020, methodology): “Searching Inspired by Strobelt et al. (2017, 2018), EX BERT performs a nearest neighbor search of embeddings on a reference corpus as follows.”
Visualizing Attention in Transformer-Based Language models (2019, methodology): “Finally, we would like to enable users to intervene in the model, either by modifying attention (Strobelt et al., 2018) or manipulating individual neurons (Bau et al.”
VizSeq: a visual analysis toolkit for text generation tasks (2019, unknown): “In NLP, softwares have been developed to interpret model parameters (e.g. attention weights) and inspect prediction generation process: LM (Rong and Adar, 2016), OpenNMT visualization tool (Klein et al., 2018) and Seq2Seq (Strobelt et al., 2019).”
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference (2019, methodology): “For example, we developed Seq2seq-Vis [73], which allows users to change inputs and constrain the inference of a translation model in order to pinpoint what part of the model is responsible for an error.”
Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “Seq2seq-Vis [235] is similar to LSTMVis but it focuses on sequence-to-sequence models, also known as encoder-decoder models, for automatic translation of texts.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Attention and gating are frequently used in NLP models, and there have been plenty of works aiming to understand the model through these weights, such as Rollout [2], Seq2Seq-Vis [157] and others [61, 158], or to investigate the reliability of using them as explanations [82, 148, 177].”
Explainability for Large Language Models: A Survey (2023, background): “In other words, explainability acts as a debugging aid to quickly advance model performance on downstream tasks (Strobelt et al., 2018; Bastings et al., 2022; Yuksekgonul et al., 2023).”
BERT Learns (and Teaches) Chemistry (2020, methodology): “Different techniques have been developed to better interpret attention maps, such as attention matrix heatmaps [2, 27, 25] and bipartite graph representations [17, 30].”
TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP (2019, methodology): “, 2020) explore supervised model knowledge in vision, while NLP methods like Seq2Seq-Vis (Strobelt et al., 2019) compare model behavior using many per-instance explanations.”
Relating Neural Text Degeneration to Exposure Bias (2021, result): “Similar phenomenon was also observed in LSTM language models (Strobelt et al., 2018).”
SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization (2021, methodology): “The CSI framework (Gehrmann et al., 2019) and the Seq2SeqVis (Strobelt et al., 2019) tool align the source document and summary, but use model-speciﬁc attention mechanisms.”