Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

By Sebastian Gehrmann, Hendrik Strobelt, Robert Krüger, Hanspeter Pfister, Alexander M. Rush. 2019

64 citations recorded by the source corpus.

Automation of tasks can have critical consequences when humans lose agency over decision processes. Deep learning models are particularly susceptible since current black-box approaches lack explainable reasoning. We argue that both the visual interface and model structure of deep learning systems need to take into account interaction design. We propose a framework of collaborative semantic inference (CSI) for the co-design of interactions and models to enable visual collaboration between humans and algorithms. The approach exposes the intermediate reasoning process of models which allows semantic interactions with the visual metaphors of a problem, which means that a user can both understand and control parts of the model reasoning process. We demonstrate the feasibility of CSI with a co-designed case study of a document summarization system.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, methodology): “Model debiasing is done through manipulation of the training data (Kaushik, Hovy, and Lipton 2019), by altering the training process (Huang et al. 2019) or by changing the model (Gehrmann et al. 2019).”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “…data (e.g., Lu et al., 2018; Hall Maudslay et al., 2019; Zhao et al., 2018a; Kaushik et al., 2019), the training process (e.g., Huang et al., 2019; Qian et al., 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al., 2019; Gehrmann et al., 2019) to ensure counterfactual fairness.”
Investigating Gender Bias in Language Models Using Causal Mediation Analysis (2020, background): “, 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al., 2019; Gehrmann et al., 2019) to ensure counterfactual fairness.”
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets (2021, background): “This follows a rich line of research in human-model collaborative interfaces, in which models generate draft outputs that humans edit [22, 10, 28, 13, 24, 23].”
TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP (2019, methodology): “While for ‘decision understanding’ methods [3, 4], input (token) importance is related to prediction strength, we instead provide ‘model understanding’ [10] by recording input token importance for neuron activation strength.”
Sequence-to-Sequence Learning with Latent Neural Grammars (2021, unknown): “Insofar as grammars and other models with symbolic components are able to better surface model decisions than standard approaches, they may have a role in developing more controllable and interpretable models, particularly in the context of collaborative human-machine systems [40].”
SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization (2021, methodology): “The CSI framework (Gehrmann et al., 2019) and the Seq2SeqVis (Strobelt et al., 2019) tool align the source document and summary, but use model-speciﬁc attention mechanisms.”
Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems (2021, background): “current neural networks, deep reinforcement learning and physics simulations [4], and (3) controllable machine capabilities for generating or summarizing when working with text [5, 6].”