Rationale-Augmented Convolutional Neural Networks for Text Classification

By Ye Zhang, Iain James Marshall, Byron C. Wallace. 2016

172 citations recorded by the source corpus.

We present a new Convolutional Neural Network (CNN) model for text classification that jointly exploits labels on documents and their constituent sentences. Specifically, we consider scenarios in which annotators explicitly mark sentences (or snippets) that support their overall document categorization, i.e., they provide rationales. Our model exploits such supervision via a hierarchical approach in which each document is represented by a linear combination of the vector representations of its component sentences. We propose a sentence-level convolutional model that estimates the probability that a given sentence is a rationale, and we then scale the contribution of each sentence to the aggregate document representation in proportion to these estimates. Experiments on five classification datasets that have document labels and associated rationales demonstrate that our approach consistently outperforms strong baselines. Moreover, our model naturally provides explanations for its predictions.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, methodology): “Proposed methods use explanations to constrain or regularize the learned model (Zaidan et al., 2007; Small et al., 2011; Ba et al., 2015; Zhang et al., 2016; Srivastava et al., 2017; Andreas et al., 2018; Liang et al., 2020), to”
Rationalizing Neural Predictions (2016, neutral): “Finally, we contrast our work with rationale-based classification (Zaidan et al., 2007; Marshall et al., 2015; Zhang et al., 2016) which seek to improve prediction by relying on richer annotations in the form of human-provided rationales.”
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations (2017, methodology): “exists work on incorporating human input into decision boundaries in the form of annotator rationales [Zaidan et al., 2007; Donahue and Grauman, 2011; Zhang et al., 2016].”
ERASER: A Benchmark to Evaluate Rationalized NLP Models (2019, background): “Zhang et al. (2016) proposed a rationale-augmented Convolu-tional Neural Network (CNN) for text classiﬁcation, explicitly trained to identify sentences supporting categorizations.”
Interpretable Neural Predictions with Differentiable Binary Variables (2019, methodology): “Our work is different from approaches that aim to improve clas-siﬁcation using rationales as an additional input (Zaidan et al., 2007; Zaidan and Eisner, 2008; Zhang et al., 2016).”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “[145] Ye Zhang, Iain Marshall, and Byron C.”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “One possibility for achieving this is to ask the model to generate explanations along with its primary prediction [353, 358].”
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases (2021, methodology): “Rationale extraction by construction: Contrary to earlier work that required supervision in the form of human-annotated rationales (Zaidan et al., 2007; Zhang et al., 2016), Lei et al.”
ToxCCIn: Toxic Content Classification with Interpretability (2021, methodology): “Zhang et al. (2016) proposed a CNN model that exploits document labels and associated rationales for text classification.”
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length (2021, background): “Our results show that for NLI, it is useful to first determine task-relevant words, suggesting future directions along the lines of Zhang et al. (2016b); Perez et al. (2019).”
A Survey of Human-in-the-loop for Machine Learning (2021, methodology): “Similar work has been done via different ways of combining human feedback and neural network [82].”
Learning Credible Deep Neural Networks with Rationale Regularization (2019, methodology): “Besides, rationales are augmented to the training process of CNN models [14], linear classification model [5], and SVMs [17].”
Toward systematic review automation: a practical guide to using machine learning tools in research synthesis (2019, background): “Many state-of-the-art systems use more complex models than logistic regression (and in particular more sophisticated methods for representing documents [13], obtaining coefficients [14], or both [15]).”
Deriving Machine Attention from Human Rationales (2018, background): “C NN ) (Zhang et al., 2016).”