Evidence Sentence Extraction for Machine Reading Comprehension

By Hai Wang, Dian Yu, Kai Sun, Jianshu Chen, Dong Yu, Dan Roth, David A. McAllester. 2019

58 citations recorded by the source corpus.

Remarkable success has been achieved in the last few years on some limited machine reading comprehension (MRC) tasks. However, it is still difficult to interpret the predictions of existing MRC models. In this paper, we focus on extracting evidence sentences that can explain or support the answers of multiple-choice MRC tasks, where the majority of answer options cannot be directly extracted from reference documents. Due to the lack of ground truth evidence sentence labels in most cases, we apply distant supervision to generate imperfect labels and then use them to train an evidence sentence extractor. To denoise the noisy labels, we apply a recently proposed deep probabilistic logic learning framework to incorporate both sentence-level and cross-sentence linguistic indicators for indirect supervision. We feed the extracted evidence sentences into existing MRC models and evaluate the end-to-end performance on three challenging multiple-choice MRC datasets: MultiRC, RACE, and DREAM, achieving comparable or better performance than the same models that take as input the full reference document. To the best of our knowledge, this is the first work extracting evidence sentences for multiple-choice MRC.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, methodology): “In this context, the GLUE benchmark (organized by some of the same authors as this work, short for General Language Understanding Evaluation; Wang et al., 2019) has become a prominent evaluation framework and leaderboard for research towards general-purpose language understanding technologies.”
Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, methodology): “(Wang et al., 2019) apply distant supervision to generate imperfect labels and then use them to train a neural evidence extractor.”
Multi-hop Question Answering via Reasoning Chains (2019, methodology): “Wang et al. (2019) apply distant supervision to generate labels and uses them to train a neural sentence extractor.”
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021, methodology): “To this end, we focus on head combinations, where os is expressive enough to explain the outputs of ot, but unlike most prior work aiming to explain by examining model outputs (Perez et al., 2019; Schuff et al., 2020; Wang et al., 2019), os is not explicitly trained for this purpose.”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “These collected sentence annotations can be considered sentence-level highlights, as they point to the sentences in a document that justify each task label (Wang et al., 2019b).”
Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents (2019, methodology): “The study in (Wang et al. 2019) designed an evidence extractor for QA tasks based on Deep Probabilistic Logic, while our model is based on GNN by taking the interaction among sentences within or across documents into account.”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, methodology): “Other works have explicitly focused on training sentence selection components for QA models (Min et al., 2018; Lin et al., 2018; Wang et al., 2019).”