Globally Normalized Reader

By Jonathan Raiman, John Miller. 2017

37 citations recorded by the source corpus.

Rapid progress has been made towards question answering (QA) systems that can extract answers from text. Existing neural approaches make use of expensive bi-directional attention mechanisms or score all possible answer spans, limiting scalability. We propose instead to cast extractive QA as an iterative search problem: select the answer’s sentence, start word, and end word. This representation reduces the space of each search step and allows computation to be conditionally allocated to promising search paths. We show that globally normalizing the decision process and back-propagating through beam search makes this representation viable and learning efficient. We empirically demonstrate the benefits of this approach using our model, Globally Normalized Reader (GNR), which achieves the second highest single model performance on the Stanford Question Answering Dataset (68.4 EM, 76.21 F1 dev) and is 24.7x faster than bi-attention-flow. We also introduce a data-augmentation method to produce semantically valid examples by aligning named entities to a knowledge base and swapping them with new entities of the same type. This method improves the performance of all models considered in this work and is of independent interest for a variety of NLP tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, methodology): “Note that Raiman & Miller (2017) recently proposed to accelerate reading comprehension by avoiding bi-directional attention and making computation conditional on the search beams.”
Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, background): “The task of selecting sentences to answer the question has been studied across several question-answering (QA) datasets, by modeling the relevance between a sentence and the question (Min et al. 2018b; Wang et al. 2019; Choi et al. 2017; Raiman and Miller 2017; Wang et al. 2018a).”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “popularity (Shen et al., 2017; Raiman and Miller, 2017; Min et al., 2018; Das et al., 2019), where a few (5-10) documents relevant to the question are retrieved and then a deep neural model finds the answer in the document.”
Efficient and Robust Question Answering from Minimal Context over Documents (2018, neutral): “We compare with GNR (Raiman and Miller, 2017), FusionNet (Huang et al.”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “2019), entities of the same type (Raiman and Miller, 2017; Dai and Adel, 2020) or words that share the same morphology (Gulordava et al.”
An Analysis of Simple Data Augmentation for Named Entity Recognition (2020, background): “Mention replacement: Raiman and Miller (2017) augment a question answering training set using an external knowledge base.”