Explaining Recurrent Neural Network Predictions in Sentiment Analysis
By Leila Arras, Grgoire Montavon, K. Müller, Wojciech Samek. 2017
385 citations recorded by the source corpus.
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specific propagation rule applicable to multiplicative connections as they arise in recurrent network architectures such as LSTMs and GRUs. We apply our technique to a word-based bi-directional LSTM model on a five-class sentiment prediction task, and evaluate the resulting LRP relevances both qualitatively and quantitatively, obtaining better results than a gradient-based related method which was used in previous work.
Subfields: Artificial Intelligence, Sentiment & Opinion Analysis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Wojciech Samek
- K. Müller
- Yonatan Belinkov
- Yoshua Bengio
- Grgoire Montavon
- Mohit Bansal
- Percy Liang
- Yoav Goldberg
In Sentiment & Opinion Analysis
- Wojciech Samek
- K. Müller
- Yonatan Belinkov
- Francesca Toni
- Piyawat Lertvittayakumjorn
- Grgoire Montavon
- Sebastian Lapuschkin
- Cuntai Guan
Citations in the corpus
- Analysis Methods in Neural Language Processing: A Survey (2018, background): “The important or salient features can then be visualized in selected examples (Li et al., 2016a; Aubakirova and Bansal, 2016; Sundararajan et al., 2017; Arras et al., 2017a,b; Ding et al., 2017; Mur-doch et al., 2018; Mudrakarta et al., 2018; Mon-tavon et al., 2018; Godin et al., 2018).”
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “See References [319 –345 • Some methods [272,280] classified in distinct categories (namely feature relevance for CNN and feature relevance for RNN ) in Fig.”
- A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, methodology): “For natural language processing or sentiment analysis, saliency map can also take the form of heat scores over words in texts, as demonstrated by [61] using LRP and by [62].”
- Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models (2017, methodology): “In the following we describe the LRP redistribution process for feed-forward neural networks, redistribution procedures have also been proposed for other popular models [5, 4, 20].”
- A unified view of gradient-based attribution methods for Deep Neural Networks (2017, methodology): “Our method differs from [15] by two aspects: 1) we always remove features based on their ranking and we do not assume our inputs are natural images, where local pixel coherence can be exploited to alter local patches; 2) while [15] performs this test removing most relevant features first (MoRF), we ”
- RUDDER: Return Decomposition for Delayed Rewards (2018, methodology): “uality = 1 jg bg Tj 1 1 ; (A267) with as quality threshold of = 80% and the maximum possible error as = 10 due to the reward scaling applied. quality is furthermore clipped to be within range [0;1]. PPO model. The ppo2 implementation [21] samples from the environment using multiple agents in paral”
- Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “Note that LRP rules have also been designed for models other than neural networks, in particular, Bag of Words classi fi ers, Fisher vector models, and LSTMs (more information can be found in the Supplementary Note 3 and Supplementary Table 1).”
- Understanding intermediate layers using linear classifier probes (2016, methodology): “Other research dealing with Relevance Propagation includes Arras et al. (2017) where this is applied to RNN in text.”
- Human-grounded Evaluations of Explanation Methods for Text Classification (2019, methodology): “Arras et al. (2017a) and Xiong et al. (2018) used relevance scores generated by explanation methods to construct document vectors by weighted-averaging word vectors and checked how well traditional machine learning techniques manage these document vectors.”
- Discovering Biased News Articles Leveraging Multiple Human Annotations (2020, methodology): “We chose this setting, because these low-level annotations can give more concrete evidence of bias and can be used as ground truth for explaining our model in the future (Arras et al., 2017).”
- Toward Interpretable Machine Learning: Transparent Deep Neural Networks and Beyond (2020, methodology): “Additional propagation rules have been proposed for special layers such as min/max pooling [13], [99], [68] and LSTM blocks [11], [9].”
- Explainable AI Methods - A Brief Overview (2020, methodology): “For instance, Arras et al. [7, 6] developed meaningful LRP redistribution rules for LSTM models.”
- Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “[206] proposed a strategy to back-propagate the relevance of the neurons in the output layer back to the input layer through the two-way multiplicative interactions between lower-layer neurons of the LSTM.”
- Sanity Checks for Saliency Metrics (2019, neutral): “Similarly, if the least relevant pixels are perturbed first (Arras et al. 2017), then the classification output should change more slowly the more accurate the saliency method.”
- Benchmarking and survey of explanation methods for black box models (2021, methodology): “- lrp was introduced for feed-forward neural networks (Arras et al. 2017) and then adapted to different type ofmodels.”
- Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference (2018, background): “…deep learning-based models, mostly by visualizing the representation of words and/or hidden states, and their importances (via saliency or erasure) on shallow tasks like sentiment analysis and POS tagging (Li et al., 2016; Arras et al., 2017; Li et al., 2017; Rei and Søgaard, 2018).”