Learning Answer-Entailing Structures for Machine Comprehension

By Mrinmaya Sachan, Kumar Avinava Dubey, Eric P. Xing, Matthew Richardson. 2015

62 citations recorded by the source corpus.

Understanding open-domain text is one of the primary challenges in NLP. Machine comprehension evaluates the system’s ability to understand text through a series of question-answering tasks on short pieces of text such that the correct answer can be found only in the given text. For this task, we posit that there is a hidden (latent) structure that explains the relation between the question, correct answer, and text. We call this the answer-entailing structure; given the structure, the correctness of the answer is evident. Since the structure is latent, it must be inferred. We present a unified max-margin framework that learns to find these hidden structures (given a corpus of question-answer pairs), and uses what it learns to answer machine comprehension questions on novel texts. We extend this framework to incorporate multi-task learning on the different subtasks that are required to perform machine comprehension. Evaluation on a publicly available dataset shows that our framework outperforms various IR and neuralnetwork baselines, achieving an overall accuracy of 67.8% (vs. 59.9%, the best previously-published result.)

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016, background): “Because many of the questions require commonsense reasoning and reasoning across multiple sentences, the dataset remains quite challenging, though there has been noticeable progress (Narasimhan and Barzilay, 2015; Sachan et al., 2015; Wang et al., 2015).”
NewsQA: A Machine Comprehension Dataset (2016, methodology): “Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al.”
Joint Concept Learning and Semantic Parsing from Natural Language Explanations (2017, methodology): “Models incorporating Statements: • RTE: This uses a Textual Entailment model (based on features from Sachan et al. (2015)) that computes a score for aligning of each statement to the text of each email.”
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task (2016, background): “Up to now, the best solutions (Sachan et al., 2015; Wang et al., 2015) are still heavily relying on manually curated syntactic/semantic features, with the aid of additional knowledge (e.”
Multi-Perspective Context Matching for Machine Comprehension (2016, methodology): “…dataset prevents researchers from building end-to-end deep neural network models, and the state-of-the-art performances are still dominated by the methods highly relying on hand-crafted features (Sachan et al., 2015; Wang and McAllester, 2015) or employing additional knowledge (Wang et al., 2016a).”
What Makes Reading Comprehension Questions Easier? (2018, background): “…of our heuristics: When we regard the MRC task as recognizing textual entailment (RTE) (Dagan et al., 2006), the task requires the reader to construct one or more premises from the context and then form the most reasonable hypothesis from the question and candidate answer (Sachan et al., 2015).”
Machine Comprehension with Syntax, Frames, and Semantics (2015, neutral): “We also compare to concurrently-published results (Narasimhan and Barzilay, 2015; Sachan et al., 2015).”
A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data (2016, methodology): “In the multiple-choice case, candidate answers are predefined, but candidate answers may also be undefined yet restricted (e.g., to yes, no, or any noun phrase in the text) (Sachan et al., 2015).”
Science Question Answering using Instructional Materials (2016, methodology): “, 2014), QA (Berant et al., 2013; Yih et al., 2013; Yao and Van Durme, 2014; Sachan et al., 2015), etc.”
Easy Questions First? A Case Study on Curriculum Learning for Question Answering (2016, methodology): “models: (a) an alignment-based approach (Sachan et al., 2015) for machine comprehension – a reading comprehension task (Richardson et al.”
Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds (2018, methodology): “, 2004) and methods that build on extracted structured representations from both the question and the input text (Sachan et al., 2015; Sachan and Xing, 2016; Khot et al., 2017; Khashabi et al., 2018b).”
Evidence Sentence Extraction for Machine Reading Comprehension (2019, background): “…such as coreference resolution, frame semantics, and discourse relations is widely used to improve machine comprehension (Wang et al., 2015; Sachan et al., 2015; Narasimhan and Barzilay, 2015; Sun et al., 2018) especially when there are only hundreds of documents available in a dataset…”
Attention-Based Convolutional Neural Network for Machine Comprehension (2016, methodology): “Representative work includes Barzilay, 2015; Sachan et al. , 2015; Wang and McAllester, 2015; Smith et al. , 2015].”
Self-Training for Jointly Learning to Ask and Answer Questions (2018, background): “Various approaches (Sachan et al., 2015; Wang et al., 2015; Sachan and Xing, 2016b; Sachan et al., 2016; Narasimhan and Barzilay, 2015) have been proposed for solving this task.”
From Textbooks to Knowledge: A Case Study in Harvesting Axiomatic Knowledge from Textbooks to Solve Geometry Problems (2017, background): “There is a growing body of work in solving standardized tests such as reading comprehensions (Richardson et al., 2013; Sachan et al., 2015, inter alia), science question answering (Schoenick et al., 2016; Sachan et al., 2016, inter alia), algebra word problems (Kushman et al., 2014, inter alia),…”
Hierarchical Attention Flow for Multiple-Choice Reading Comprehension (2018, methodology): “Existing models (Richardson, Burges, and Renshaw 2013; Sachan et al. 2015; Narasimhan and Barzilay 2015; Wang et al. 2015; Smith et al. 2015; Trischler et al. 2016; Yin, Ebert, and Schütze 2016; Lai et al. 2017) for multiple-choice reading comprehension score each candidate option independently.”