LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation
By Pankaj Gupta, Hinrich Schütze. 2018
18 citations recorded by the source corpus.
Recurrent neural networks (RNNs) are temporal networks and cumulative in nature that have shown promising results in various natural language processing tasks. Despite their success, it still remains a challenge to understand their hidden behavior. In this work, we analyze and interpret the cumulative nature of RNN via a proposed technique named as Layer-wIse-Semantic-Accumulation (LISA) for explaining decisions and detecting the most likely (i.e., saliency) patterns that the network relies on while decision making. We demonstrate (1) LISA: “How an RNN accumulates or builds semantics during its sequential processing for a given text example and expected response” (2) Example2pattern: “How the saliency patterns look like for each category in the data according to the network in decision making”. We analyse the sensitiveness of RNNs about different inputs to check the increase or decrease in prediction scores and further extract the saliency patterns learned by the network. We employ two relation classification datasets: SemEval 10 Task 8 and TAC KBP Slot Filling to explain RNN predictions via the LISA and example2pattern.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yonatan Belinkov
- James R. Glass
- Henry Nunoo-Mensah
- Wenyu Chen
- Francisca Adoma Acheampong
- Nadir Durrani
- Hassan Sajjad
- Fahim Dalvi
In Safety, Robustness & Interpretability
- Yonatan Belinkov
- James R. Glass
- Sebastian Gehrmann
- Michael R Douglas
- Percy Liang
- Ellie Pavlick
- Jesse Vig
- Ivan Titov
Citations in the corpus
- Analysis Methods in Neural Language Processing: A Survey (2018, background): “Gupta and Schütze (2018) inspected how information is accumulated in RNNs towards a prediction, and associated peaks in prediction scores with important input segments.”
- Transformer models for text-based emotion detection: a review of BERT-based approaches (2021, background): “However, they highlighted a better neural architecture capable of extracting salient fragments/keywords could improve performance (Gupta and Schütze 2018).”