Rationalizing Neural Predictions

By Tao Lei, Regina Barzilay, Tommi S. Jaakkola. 2016

880 citations recorded by the source corpus.

Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications -- rationales -- that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Measuring and Improving BERT’s Mathematical Abilities by Predicting the Order of Reasoning. (2021, background): “…et al., 2014; Mnih et al., 2014; Gulcehre et al., 2016; Malinowski et al., 2018; Xu and Saenko, 2016; Yang et al., 2016), phrase selection (Lei et al., 2016), distillation into programs (Hajipour et al., 2020), or decision trees (Alaniz and Akata, 2019) can potentially increase the…”
Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems (2017, methodology): “Lei et al. (2016), jointly modeled both a classiﬁcation decision, and the selection of the most relevant subsection of a document for making the classiﬁcation decision.”
Finding Generalizable Evidence by Learning to Convince Q&A Models (2019, methodology): “Inspired by work in interpreting neural networks (Lei et al., 2016), we have agents find a subset of the passage (i.”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “Treating sentence selection as a latent variable has been explored in classiﬁ-cation (Yessenalina et al., 2010; Lei et al., 2016), however, to our knowledge, has not been applied for question answering.”
ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, methodology): “By design, we ensure that the generated proof is solely used for determining the label of claim, making it a faithful by construction model (Lei et al., 2016; Jain et al., 2020).”
Training Classifiers with Natural Language Explanations (2018, methodology): “Finally, while we have used natural language explanations as input to train models, they can also be output to interpret models (Krening et al., 2017; Lei et al., 2016).”
Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction (2022, background): “We call this step rationalization because it could potentially give us the rationale that explains an outcome (Lei et al., 2016).”
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, background): “(yes or no) This question is designed to test for clarity, aligning with characteristics such as coherence (Lei et al., 2016) and human-likeness and understandability (Ehsan et al., 2019).”
Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Few-Shot Self-Rationalization A standard approach to creating explanations in the form of highlights is the select-then-predict method (Lei et al., 2016) that does not use any human-author input highlights.”
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, methodology): “Another recent line of work explores collecting rationales (Lei et al., 2016) through expert annotations (Zaidan and Eisner, 2008).”
Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “the input and is sufficient to make a prediction on its own, without the rest of the input (Lei et al., 2016).”
Benefits of Intermediate Annotations in Reading Comprehension (2020, background): “Especially relevant is work on supervising models using explanations (Ross et al., 2017), which, similar to our annotations, identify parts of the input that are important for prediction (Lei et al., 2016; Ribeiro et al., 2016).”
Explain Yourself! Leveraging Language Models for Commonsense Reasoning (2019, background): “Lei et al. (2016) proposed an approach for rationale generation for sentiment analysis by highlighting complete phrases in the input text that by itself is sufﬁcient to predict the desired output.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “For example, Lei et al. (2016) deﬁned a generator that learns a distribution over text fragments as candidate rationales for justifying predictions, evaluated on sentiment analysis.”
Towards A Rigorous Science of Interpretable Machine Learning (2017, background): “…interpretability in the context of an application: if the system is useful in either a practical application or a simpliﬁed version of it, then it must be somehow interpretable (e.g. Ribeiro et al. [2016], Lei et al. [2016], Kim et al. [2015a], Doshi-Velez et al. [2015], Kim et al. [2015b]).”
Attention is not Explanation (2019, background): “Such instantiations explicitly select (modest) subsets of inputs to be considered when making a prediction, which are then by construction responsible for model output (Lei et al., 2016; Peters et al., 2018).”
Attention is not not Explanation (2019, methodology): “Generally, many accept the notion of extractive methods such as Lei et al. (2016), in which explanations come directly from the input itself (as in attention), as plausible.”
Understanding Neural Networks through Representation Erasure (2016, methodology): “Task, Dataset and Training Inspired by recent visualization work from Lei et al. (2016), we focus on the task of document-level aspect rating prediction (Tang et al., 2015a; Tang et al., 2015b).”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “Decision sets, rule sets [83], [84] X 7 Verbal Encoder-generator framework [85] 7 X Filter Attribute Probability Density Function [86] 7 7”
EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction (2019, methodology): “Such methods are speciﬁc to text data select and rationales, i.e. parts of text, on which the model bases its prediction, see [15, 27] and very recently [5].”