Multi-Mention Learning for Reading Comprehension with Neural Cascades

By Swabha Swayamdipta, Ankur P. Parikh, Tom Kwiatkowski. 2017

35 citations recorded by the source corpus.

Reading comprehension is a challenging task, especially when executed across longer or across multiple evidence documents, where the answer is likely to reoccur. Existing neural architectures typically do not scale to the entire evidence, and hence, resort to selecting a single passage in the document (either via truncation or other means), and carefully searching for the answer within that passage. However, in some cases, this strategy can be suboptimal, since by focusing on a specific passage, it becomes difficult to leverage multiple mentions of the same answer throughout the document. In this work, we take a different approach by constructing lightweight models that are combined in a cascade to find the answer. Each submodel consists only of feed-forward networks equipped with an attention mechanism, making it trivially parallelizable. We show that our approach can scale to approximately an order of magnitude larger evidence documents and can aggregate information at the representation level from multiple mentions of each answer candidate across the document. Empirically, our approach achieves state-of-the-art performance on both the Wikipedia and web domains of the TriviaQA dataset, outperforming more complex, recurrent architectures.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Question Answering & Reading Comprehension

Citations in the corpus

Simple and Effective Multi-Paragraph Reading Comprehension (2017, background): “92 Neural Cascade (Swayamdipta et al., 2017) 53.”
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “Most work concatenates passages into a single sequence (Swayamdipta et al., 2018; Yang et al., 2018; Song et al., 2018) or reads each in parallel (Clark and Gardner, 2018; Alberti et al., 2019; Min et al., 2019b; Wang et al., 2019).”
A Discrete Hard EM Approach for Weakly Supervised Question Answering (2019, methodology): “Some models are trained with maximum marginal likelihood (MML) (Kadlec et al., 2016; Swayamdipta et al., 2018; Clark and Gardner, 2018; Lee et al., 2019), but it is unclear if it gives a meaningful improvement over the heuristics.”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, background): “full Wikipedia document) (Swayamdipta et al., 2018; Clark & Gardner, 2018) and in open domain settings (Wang et al.”
Efficient and Robust Question Answering from Minimal Context over Documents (2018, background): “For TriviaQA(Wikipedia), they are Neural Casecades (Swayamdipta et al., 2018), Reading Twice for Natural Language Understanding (Weissenborn, 2017) and Mnemonic Reader (Hu et al.”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, neutral): “Swayamdipta et al. (2017) attempted cascaded non-recurrent small networks on extractive QA, resulting a scalable, parallelizable model.”
Coreferential Reasoning Learning for Language Representation (2020, unknown): “It is an approach widely used in question answering (Kadlec et al., 2016; Swayamdipta et al., 2018; Clark and Gardner, 2018) designed to handle multiple answers.”
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension (2018, background): “, 2018; Nogueira and Cho, 2017) or cascades (Swayamdipta et al., 2018).”
A dataset and baselines for sequential open-domain question answering (2018, background): “Previous work considers improving that base framework itself (Clark and Gardner, 2018; Swayamdipta et al., 2018, inter alia) but retains the assumption of answering individual questions.”