Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?

By Danish Pruthi, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary Lipton, Graham Neubig, William W. Cohen. 2020

123 citations recorded by the source corpus.

While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this work, we introduce a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. Crucially, the explanations are available to the student during training, but are not available at test time. Compared with prior proposals, our approach is less easily gamed, enabling principled, automatic, model-agnostic evaluation of attributions. Using our framework, we compare numerous attribution methods for text classification and question answering, and observe quantitative differences that are consistent (to a moderate to high degree) across different student model architectures and learning strategies.1

Subfields: Artificial Intelligence, Computer Science Applications, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Evaluation & Benchmarks

Citations in the corpus

Large Language Models Can Self-Improve (2022, neutral): “Majority Voting by answer ···”
Training Language Models with Language Feedback (2022, methodology): “Prior work explores ways of using explanations to train text classiﬁcation models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021; Lampinen et al., 2022, inter alia ).”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, background): “Adopting terminology from Pruthi et al. (2020), we refer to a teacher giving explanations to a student who is learning a task.”
Training Language Models with Natural Language Feedback (2022, methodology): “Prior work explores ways of using explanations to train text classiﬁcation models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Natural Language Inference with a Human Touch: Using Human Explanations to Guide Model Attention (2021, methodology): “…to improve performance on a range of domains, including commonsense reasoning (Rajani et al., 2019), relation extraction (Murty et al., 2020), visual clas-siﬁcation tasks (Liang et al., 2020; Mu et al., 2020), sentiment analysis (Pruthi et al., 2020) and question answering (Pruthi et al., 2020).”
Improving Code Generation by Training with Natural Language Feedback (2023, background): “…other works that have explored supervision via natural language in other ways, such as via explanations (Camburu et al., 2018; Hase & Bansal, 2021; Pruthi et al., 2021; Lampinen et al., 2022, inter alia ) and as part of RL systems (Fidler et al., 2017; Luketina et al., 2019; Lin et al., 2020,…”
Training Language Models with Language Feedback at Scale (2023, methodology): “Prior work explores ways of using explanations to train text classification models, with mixed results (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase & Bansal, 2021; Lampinen et al., 2022, inter alia ).”
Supervising Model Attention with Human Explanations for Robust Natural Language Inference (2021, methodology): “Our methods and the Pruthi et al. (2020) method were tested over the same 25 random seeds, while the highly computationally expensive LIREx-adapted approach was evaluated over 5 random seeds. top half of Table 1).”
REV: Information-Theoretic Evaluation of Free-Text Rationales (2022, background): “Evaluations on extractive rationales have been well studied, generally from two perspectives — faithfulness and plausibility (DeYoung et al., 2020; Pruthi et al., 2022; Chan et al., 2022b).”
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021, background): “…feature attribution —identifying important tokens in the input span, e.g. via saliency maps (Li et al., 2016; Ribeiro et al., 2016); and (2) instance attribution —explaining the model decisions as a function of inﬂuential training data (Koh and Liang, 2017; Han et al., 2020; Pruthi et al., 2020b).”
Learning from Natural Language Feedback (2024, methodology): “Prior work explores ways of using explanations to train text classification models, with mixed re-sults (Camburu et al., 2018; Stacey et al., 2021; Pruthi et al., 2021; Wiegreffe et al., 2021; Hase and Bansal, 2021, inter alia ).”
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes (2023, methodology): “, 2019); it can be used to improve overall model performance (Zaidan et al., 2007; Zhang et al., 2016; Camburu et al., 2018; Pruthi et al., 2022); and human rationales can be used as gold standard labels to make models more interpretable by generating similar rationales (Wiegreffe et al.”
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning (2021, unknown): “Thus, following previous works (Zhang* et al., 2020; Sellam et al., 2020; Pruthi et al., 2020), we propose an automatic model-based metric that given a belief-graph pair, predicts the label between incorrect, support, and counter.”