Interpretable and Steerable Sequence Learning via Prototypes

By Yao Ming, Panpan Xu, Huamin Qu, Liu Ren. 2019

168 citations recorded by the source corpus.

One of the major challenges in machine learning nowadays is to provide predictions with not only high accuracy but also user-friendly explanations. Although in recent years we have witnessed increasingly popular use of deep neural networks for sequence modeling, it is still challenging to explain the rationales behind the model outputs, which is essential for building trust and supporting the domain experts to validate, critique and refine the model. We propose ProSeNet, an interpretable and steerable deep sequence model with natural explanations derived from case-based reasoning. The prediction is obtained by comparing the inputs to a few prototypes, which are exemplar cases in the problem domain. For better interpretability, we define several criteria for constructing the prototypes, including simplicity, diversity, and sparsity and propose the learning objective and the optimization procedure. ProSeNet also provides a user-friendly approach to model steering: domain experts without any knowledge on the underlying model or parameters can easily incorporate their intuition and experience by manually refining the prototypes. We conduct experiments on a wide range of real-world applications, including predictive diagnostics for automobiles, ECG, and protein sequence classification and sentiment analysis on texts. The result shows that ProSeNet can achieve accuracy on par with state-of-the-art deep learning models. We also evaluate the interpretability of the results with concrete case studies. Finally, through user study on Amazon Mechanical Turk (MTurk), we demonstrate that the model selects high-quality prototypes which align well with human knowledge and can be interactively refined for better interpretability without loss of performance.

Subfields: Artificial Intelligence, Materials Chemistry, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “In the sequence modeling domain (such as natural language processing), Ming et al. (2019) and Hong et al. (2020) took the concepts in Li et al. (2018) and Chen et al. (2019), and integrated prototype learning into recurrent neural networks for modeling sequential data.”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, methodology): “While [29] presents as rule of thumb that the number of prototypes should be 2-3 times the number of classes, ProtoPNet [11] uses 10 prototypes per class.”
Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition (2019, methodology): “…models can be constructed for computer vision and time-Harvard series analysis (e.g., Chen et al., 2019; Y. Li et al., 2017; O. Li et al., 2018; Ming et al., 2019), then the standard should be changed from the assumption that interpretable models do not exist, to the assumption that they do ,…”
This looks like that: deep learning for interpretable image recognition (2018, neutral): “[28] recently took the concepts in [23] and the preprint of an earlier version of this work, which both involve integrating prototype learning into CNNs for image recognition, and used these concepts to develop prototype learning in recurrent neural networks for modeling sequential data.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “More works related to prototypes or exemplars for interpretations can be found in [23, 26, 103, 116].”
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, methodology): “This can be seen in the work of Al-varez et. al [ 3] for basis concept learning and Kim et. al [30, 42] for case-based reasoning and prototype learning.”
Interpretable Sequence Classification Via Prototype Trajectory (2020, methodology): “The original paper (Ming et al., 2019) validates ProSeNet only on paragraphs shorter than 25 words.”
ProtoSiTex: Learning Semi-Interpretable Prototypes for Multi-label Text Classification (2025, unknown): “To comprehensively evaluate ProtoSiTex, we compare it against a diverse set of baselines and SOTA models, as summarized in Table III.”