Teaching Machine Comprehension with Compositional Explanations

By Qinyuan Ye, Xiao Huang, Xiang Ren. 2020

36 citations recorded by the source corpus.

Advances in machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated examples in the form of (question, paragraph, answer) triples. In contrast, humans are typically able to generalize with only a few examples, relying on deeper underlying world knowledge, linguistic sophistication, and/or simply superior deductive powers. In this paper, we focus on “teaching” machines reading comprehension, using a small number of semi-structured explanations that explicitly inform machines why answer spans are correct. We extract structured variables and rules from explanations and compose neural module teachers that annotate instances for training downstream MRC models. We use learnable neural modules and soft logic to handle linguistic variation and overcome sparse coverage; the modules are jointly optimized with the MRC model to improve final performance. On the SQuAD dataset, our proposed method achieves 70.14% F1 score with supervision from 26 explanations, comparable to plain supervised learning using 1,100 labeled instances, yielding a 12x speed up.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, background): “On the other hand, several works [15, 47, 51] argue that natural language explanations rather than numeric labels as human feedback provide a richer and more efficient source of supervision for training.”
Explaining Answers with Entailment Trees (2021, background): “, 2019), or the syntactic pattern used to locate the answer (Ye et al., 2020; Hancock et al., 2018).”
On the Diversity and Limits of Human Explanations (2021, background): “Reference Task Questions and guidelines Lamm et al. (2020) question an- swering referential equality, we cannot infer the exact question Ye et al. (2020) question an- swering Please read carefully to get accepted!”
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020, background): “…a collection of text snippets supporting an answer (DeYoung et al., 2020), an attention map over a passage (Seo et al., 2016), a synthesized phrase connecting question and answer (Rajani et al., 2019), or the syntactic pattern used to locate the answer (Ye et al., 2020; Hancock et al., 2018).”
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning (2021, background): “6), and (3) unlike semi-structured templates (Ye et al., 2020; Mostafazadeh et al., 2020) or extractive rationales (Zaidan et al., 2007; Lei et al., 2016; Yu et al., 2019; DeYoung et al., 2020), they allow for more ﬂexi-bility and expressiveness.”
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021, background): “semi-structured text (Ye et al., 2020) to chain of facts (Khot et al.”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “Ye et al. (2020) collect semi-structured explanations for N ATURAL Q UESTIONS (Kwiatkowski et al., 2019) and SQ U AD (Rajpurkar et al., 2016).”