Large-scale Simple Question Answering with Memory Networks

By Antoine Bordes, Nicolas Usunier, Sumit Chopra, Jason Aaron Edward Weston. 2015

745 citations recorded by the source corpus.

Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions. This paper studies the impact of multitask and transfer learning for simple question answering; a setting for which the reasoning required to answer is quite easy, as long as one can retrieve the correct evidence given a question, which can be difficult in large-scale conditions. To this end, we introduce a new dataset of 100k questions that we use in conjunction with existing benchmarks. We conduct our study within the framework of Memory Networks (Weston et al., 2015) because this perspective allows us to eventually scale up to more complex reasoning, and show that Memory Networks can be successfully trained to achieve excellent performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (2017, background): “Proposed datasets (Cai and Yates, 2013; Berant et al., 2013; Bordes et al., 2015) are either limited in scale or in the complexity of questions, and can only retrieve facts covered by the KB.”
CoQA: A Conversational Question Answering Challenge (2018, background): “Another common knowledge source is machine-friendly databases which organize world facts in the form of a table or a graph (Berant et al., 2013; Pasupat and Liang, 2015; Bordes et al., 2015; Saha et al., 2018; Talmor and Berant, 2018).”
Reformer: The Efficient Transformer (2020, methodology): “The original implementation of memory networks (Weston et al., 2014) and later work on scaling it (Bordes et al., 2015; Chandar et al., 2016) used memory with size in the millions.”
Towards a Better Metric for Evaluating Question Generation Systems (2018, background): “…for document Question Answering (QA) (Rajpurkar et al., 2016; Nguyen et al., 2016; Joshi et al., 2017; Saha et al., 2018a) knowledge base driven QA (Bordes et al., 2015; Saha et al., 2018b) and Visual QA (Antol et al., 2015; Johnson et al., 2017) has enabled the development of end-to-end…”
Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering (2017, background): “The task is to produce the answer to a question by exploiting resources such as documents (Voorhees, 1999), webpages (Kwok et al., 2001) or structured knowledge bases (Berant et al., 2013; Bordes et al., 2015; Yu et al., 2017).”
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine (2017, methodology): “In 2015, Bordes et al. (2015) released a large-scale dataset of 100k open-world question-answer pairs constructed from Freebase, and Hermann et al. (2015) released two datasets, each consisting of closed-world question-answer pairs automatically generated from news articles.”
Reading Wikipedia to Answer Open-Domain Questions (2017, methodology): “Bordes et al. (2015) combined We-bQuestions and SimpleQuestions using distant supervision with Freebase as the KB to give slight improvements on both datasets, although poor performance was reported when training on only one dataset and testing on the other, showing that task transfer is indeed a…”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, background): “Specific recent applications in question answering include Bordes et al. (2015), Chen et al. (2017), Lin et al. (2018), as well as Joshi et al. (2017) for building benchmark test collections.”
Denoising Distantly Supervised Open-Domain Question Answering (2018, background): “, 2001; Chen and Van Durme, 2017), structured knowledge graphs (Berant et al., 2013a; Bordes et al., 2015) or automatically extracted relational triples (Fader et al.”
Key-Value Memory Networks for Directly Reading Documents (2016, methodology): “Using SimpleQuestions (Bordes et al., 2015), an existing open-domain question answering dataset based on Freebase, we identified the subset of questions posed by human annotators that covered our question types.”
Zero-Shot Relation Extraction via Reading Comprehension (2017, methodology): “The Simple QA dataset (Bordes et al., 2015) was created by annotating questions about individual Freebase facts (e.”
Learning Through Dialogue Interactions (2016, methodology): “, 2015; Weston, 2016), using either a given database of knowledge (Bordes et al., 2015; Miller et al., 2016) or short stories (Weston et al.”
Dialog-based Language Learning (2016, background): “Question answering given either a database of knowledge [2] or short stories [29] can be considered as a simple case of dialog which is easy to evaluate.”
Visual Dialog (2016, methodology): “Some recent large-scale datasets in this domain include the 30M Factoid Question-Answer corpus [44], 100K SimpleQuestions dataset [6], DeepMind Q&A dataset [15], the 20 artificial tasks in the bAbI dataset [57], and the SQuAD dataset for reading comprehension [38].”
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations (2015, methodology): “A related approach was successfully applied by Bordes et al. (2015) to question answering about knowledge bases.”
Constructing Datasets for Multi-hop Reading Comprehension Across Documents (2017, methodology): “…QA has witnessed a surge in interest with the advent of large-scale datasets, which have been assembled based on F REEBASE (Berant et al., 2013; Bordes et al., 2015), W IKIPEDIA (Yang et al., 2015; Rajpurkar et al., 2016; Hewlett et al., 2016), web search queries (Nguyen et al., 2016), news…”
Crowdsourcing Multiple Choice Science Questions (2017, background): “…and web search user queries (Nguyen et al., 2016); for reading comprehension (RC) based on news (Hermann et al., 2015; Onishi et al., 2016), children books (Hill et al., 2015) and novels (Pa-perno et al., 2016), and for recognizing textual entailment based on image captions (Bowman et al., 2015).”