A Neural Network for Factoid Question Answering over Paragraphs

By Mohit Iyyer, Jordan L. Boyd-Graber, Leonardo Max Batista Claudino, Richard Socher, Hal Daumé Iii. 2014

375 citations recorded by the source corpus.

Text classification methods for tasks like factoid question answering typically use manually defined string matching rules or bag of words representations. These methods are ineective when question text contains very few individual words (e.g., named entities) that are indicative of the answer. We introduce a recursive neural network (rnn) model that can reason over such input by modeling textual compositionality. We apply our model, qanta, to a dataset of questions from a trivia competition called quiz bowl. Unlike previous rnn models, qanta learns word and phrase-level representations that combine across sentences to reason about entities. The model outperforms multiple baselines and, when combined with information retrieval methods, rivals the best human players.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (2017, background): “10 A number of different aspects of this problem have been carefully studied, typically using classifiers over a pre-defined set of answers (Iyyer et al., 2014) and studying incremental answering to answer as”
Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval (2015, methodology): “A selection of successful applications of this approach include sequence labeling (Turian et al., 2010), parsing (Chen and Manning, 2014), sentiment (Socher et al., 2013), question answering (Iyyer et al., 2014) and translation modeling (Gao et al., 2014a).”
Reading Wikipedia to Answer Open-Domain Questions (2017, background): “That subﬁeld has made considerable progress recently thanks to new deep learning architectures like attention-based and memory-1 augmented neural networks (Bahdanau et al., 2015; Weston et al., 2015; Graves et al., 2014) and release of new training and evaluation datasets like QuizBowl (Iyyer et al.”
Memory Networks (2014, background): “This gives a QA task on simple “stories” such as in Figure 1.”
Neural Module Networks (2015, background): “While all the applications considered in this paper involve visual question answering, the architecture is much more general, and might easily be applied to visual referring expression resolution [9, 34] or question answering about natural language texts [15].”
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (2015, methodology): “There are many different approaches to question answering: some build large knowledge bases (KBs) with open information extraction systems (Yates et al., 2007), some use neural networks, dependency trees and KBs (Bordes et al., 2012), others only sentences (Iyyer et al., 2014).”
Visual7W: Grounded Question Answering in Images (2015, background): “Recent neural network models attempt to learn end-to-end directly from questions and answers [12, 46].”
Learning to Compose Neural Networks for Question Answering (2016, background): “These include approaches that model the task directly as a multiclass classification problem (Iyyer et al., 2014), models that attempt to embed questions and answers in a shared vector space (Bordes et al., 2014) and attentional models that select words from documents sources (Hermann et al., 2015).”
Neural Programmer: Inducing Latent Programs with Gradient Descent (2015, methodology): “Neural networks have also been used for Q&A on datasets that do not require complicated arithmetic and logic reasoning (Bordes et al., 2014; Iyyer et al., 2014; Sukhbaatar et al., 2015; Peng et al., 2015; Hermann et al., 2015).”
Dynamic Memory Networks for Visual and Textual Question Answering (2016, methodology): “Neural approaches can include reasoning over knowledge bases, (Bordes et al., 2012; Socher et al., 2013a) or directly via sentences for trivia competitions (Iyyer et al., 2014).”
Modular Multitask Reinforcement Learning with Policy Sketches (2016, methodology): “Because the value function is itself unknown, an approximation must be estimated from data.”
Deep Learning for Answer Sentence Selection (2014, background): “Another line of work—clo se y related to the model presented here—is the application of recursive neural networks to fac toid question answering over paragraphs [12].”
Towards a Visual Turing Challenge (2014, background): “The p rogress in both fields has inspired researchers to build holistic architectures for challengi ng grounding [14, 15], natural language generation from image/video [16, 17, 18], image-to-sentence a lignment [19, 20, 21, 22], and recently presented question-answering problems [23, 24, 25, 26, 27]”