End-to-End Open-Domain Question Answering with BERTserini

By Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, Jimmy J. Lin. 2019

526 citations recorded by the source corpus.

We demonstrate an end-to-end question answering system that integrates BERT with the open-source Anserini information retrieval toolkit. In contrast to most question answering and reading comprehension models today, which operate over small amounts of input text, our system integrates best practices from IR with a BERT-based reader to identify answers from a large corpus of Wikipedia articles in an end-to-end fashion. We report large improvements over previous results on a standard benchmark test collection, showing that fine-tuning pretrained BERT with SQuAD is sufficient to achieve high accuracy in identifying answer spans.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “1 drops from above 80% to less than 40% (Yang et al., 2019a).”
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, background): “On the other hand, the performance of most ex-tractive models seems to peak around 10 to 20 passages (Wang et al., 2019; Yang et al., 2019).”
Distilling Knowledge from Reader to Retriever for Question Answering (2020, methodology): “This lead Nogueira & Cho (2019) and Yang et al. (2019) to use a BERT model to jointly embed documents and query, a technique known as cross-encoder.”
How Context Affects Language Models' Factual Predictions (2020, methodology): “Finally, while we have chosen DrQA as our baseline to compare to recent work, there exist several more sophisticated supervised open-domain QA models that outperform it on a variety of open-domain QA tasks [Lee et al., 2019, Yang et al., 2019a, Guu et al., 2020].”
Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, methodology): “Inspired by BERTserini (Yang et al., 2019), the ﬁnal score is a learned weighted sum of the BM25 and reader score.”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “Following this formulation, a couple of recent studies have proposed to improve the system us-ing stronger reading comprehension models (Yang et al., 2019; Wang et al., 2018), more effective learning objectives (Clark and Gardner, 2018; Min et al., 2019a; Wang et al., 2019) or paragraph reranking…”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, background): “(Hu et al., 2019) 50.2 41.9 MUPPET (Feldman & El-Yaniv, 2019) 46.2 39.3 BERTserini (Yang et al., 2019) 46.1 38.6 DENSPI-hybrid (Seo et al., 2019) 44.4 36.2 MINIMAL (Min et al., 2018) 42.5 34.7 Multi-step Reasoner (Das et al., 2019) Re-rank2hop to Entity-centric IR demonstrates that exploring entity ”
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “Open-domain question answering systems aim to answer any question a user can pose, with evidence provided by either factual text such as Wikipedia (Chen et al., 2017; Yang et al., 2019) or knowledge bases (KBs) such as Freebase (Be-rant et al., 2013; Kwiatkowski et al., 2013; Yih et al., 2015).”
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering (2019, methodology): “Effect of passage granularity : Previous work usually deﬁnes passages as articles (Chen et al., 2017), paragraphs (Yang et al., 2019), or sentences (Wang et al., 2018a; Lin et al., 2018).”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, neutral): “We additionally compare with results of a recent paper (Yang et al., 2019) that uses BERT encodings for open-domain QA, and is recently made available on arXiv.”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, methodology): “Following Yang et al. (2019), to evaluate answers in an end-to-end setup, we disregard the paragraph context from the original datasets and use only the answer spans.”
You Only Need One Model for Open-domain Question Answering (2021, background): “These score the relevance of knowledge to a given query by lexical overlaps be-tween them in a sparse representation space based on TF-IDF or BM25 (Chen et al., 2017; Wang et al., 2018; Yang et al., 2019).”
R2-D2: A Modular Baseline for Open-Domain Question Answering (2021, neutral): “Yang et al. (2019) also combined BM25 ranker and reader scores via linear combination.”
Learning Dense Representations of Phrases at Scale (2020, background): “The model can be easily adapted from documents to passages and has been studied extensively (Yang et al., 2019; Wang et al., 2019).”
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model (2019, methodology): “This direction has been further explored at a larger scale with efficient Transformer architectures (Radford et al., 2019; Devlin et al., 2019; Yang et al., 2019c).”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “…In contrast to sparse term-index IR methods, which are widely used by existing open-domain QA systems (Chen et al., 2017; Wang et al., 2018; Yang et al., 2019), recent systems (Lee et al., 2019; Guu et al., 2020; Karpukhin et al., 2020) typically uses dense passage retrieval techniques…”