Efficient and Robust Question Answering from Minimal Context over Documents

By Sewon Min, Victor Zhong, Richard Socher, Caiming Xiong. 2018

168 citations recorded by the source corpus.

Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, methodology): “They earned their living by traveling the famous Silk Road.”
How Context Affects Language Models' Factual Predictions (2020, background): “Works in this areas mostly focus on enhancing retrieval quality [Choi et al., 2017, Wang et al., 2018b,c, Lin et al., 2018, Min et al., 2018, Lee et al., 2018, 2019, Das et al., 2019, Xiong et al., 2019], improving answer aggregation [Clark and Gardner, 2018, Wang et al., 2018c, Lee et al., 2018,…”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, neutral): “(Hu et al., 2019) 50.2 41.9 MUPPET (Feldman & El-Yaniv, 2019) 46.2 39.3 BERTserini (Yang et al., 2019) 46.1 38.6 DENSPI-hybrid (Seo et al., 2019) 44.4 36.2 MINIMAL (Min et al., 2018) 42.5 34.7 Multi-step Reasoner (Das et al., 2019) Re-rank2hop to Entity-centric IR demonstrates that exploring entity ”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “…(Rajpurkar et al., 2016), open-domain phrase-level question answering has gained a great popularity (Shen et al., 2017; Raiman and Miller, 2017; Min et al., 2018; Raison et al., 2018; Das et al., 2019), where a few (5-10) documents relevant to the question are retrieved and then a deep neural…”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, neutral): “2 - MINIMAL (Min et al., 2018) 34.”
End-to-End Open-Domain Question Answering with BERTserini (2019, neutral): “2 - MINIMAL (Min et al., 2018) 34.”
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension (2020, positive): “This is in line with prior observations on lexical overlap as a predictive cue in SQuAD (Weissenborn et al., 2017; Min et al., 2018); questions with less overlap are harder to answer for any of the three models.”
What Makes Reading Comprehension Questions Easier? (2018, background): “Min et al. (2018) observed that 92% of answerable questions in SQuAD can be answered only using a single context sentence.”
Multi-hop Reading Comprehension through Question Decomposition and Rescoring (2019, background): “Most of the questions on these datasets can be answered in a single sentence (Min et al., 2018), which is a key difference from multi-hop reading comprehension.”
Multi-hop Question Answering via Reasoning Chains (2019, methodology): “Our approach resembles past models for coarse-to-fine question answering (Choi et al., 2017; Min et al., 2018; Wang et al., 2019), but explores the context in a sequential fashion and is trained to produce principled chains.”
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020, background): “Later, other proposals were able to surpass its quality of answer with a paragraph reranker (Lee et al. 2018; Wang et al. 2017) and then with a ”minimal” retriever that selects small but relevant portions of text (Min et al. 2018).”
T RANSFORMER -XH: M ULTI - HOP Q UESTION A N - SWERING WITH E X TRA H OP A TTENTION (2019, background): “The initial MRC questions are mostly single-hop, which only require single evidence sentence to answer (Min et al., 2018).”
Look at the First Sentence: Position Bias in Question Answering (2020, methodology): “2 We conduct experiments on SQuAD ( D = SQuAD) as most examples in SQuAD are answerable with a single sentence (Min et al., 2018).”
Multi-style Generative Reading Comprehension (2019, background): “The most recent models make a proper trade-off between efficiency and accuracy (Yan et al., 2018; Min et al., 2018).”