Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering
By Vikas Yadav, Rebecca Sharp, Mihai Surdeanu. 2018
16 citations recorded by the source corpus.
While increasingly complex approaches to question answering (QA) have been proposed, the true gain of these systems, particularly with respect to their expensive training requirements, can be in- flated when they are not compared to adequate baselines. Here we propose an unsupervised, simple, and fast alignment and informa- tion retrieval baseline that incorporates two novel contributions: a one-to-many alignment between query and document terms and negative alignment as a proxy for discriminative information. Our approach not only outperforms all conventional baselines as well as many supervised recurrent neural networks, but also approaches the state of the art for supervised systems on three QA datasets. With only three hyperparameters, we achieve 47% P@1 on an 8th grade Science QA dataset, 32.9% P@1 on a Yahoo! answers QA dataset and 64% MAP on WikiQA.
Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Information Systems
In Information Retrieval & Search
In Natural Language Processing
Citations in the corpus
- Improving Question Answering with External Knowledge (2019, methodology): “As shown in Table 5, the baseline based on BERT already outperforms previous state-of-the-art methods designed for subject-area QA tasks (Yadav et al., 2018; Pirtoaca et al., 2019; Ni et al., 2019; Sun et al., 2019).”
- Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, methodology): “98 > 0 1 No No Sanity Check (Yadav et al., 2018) 58.”
- Alignment over Heterogeneous Embeddings for Question Answering (2019, methodology): “Recently, Yadav et al. (2018) has shown that alignment meth-ods remain competitive for non-factoid QA.”