R2-D2: A Modular Baseline for Open-Domain Question Answering

By Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz. 2021

50 citations recorded by the source corpus.

This work presents a novel four-stage open-domain QA pipeline R2-D2 (Rank twice, reaD twice). The pipeline is composed of a retriever, passage reranker, extractive reader, generative reader and a mechanism that aggregates the final prediction from all system's components. We demonstrate its strength across three open-domain QA datasets: NaturalQuestions, TriviaQA and EfficientQA, surpassing state-of-the-art on the first two. Our analysis demonstrates that: (i) combining extractive and generative reader yields absolute improvements up to 5 exact match and it is at least twice as effective as the posterior averaging ensemble of the same models with different parameters, (ii) the extractive reader with fewer parameters can match the performance of the generative reader on extractive QA datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hamed Zamani
2. Sebastian Hofstätter
3. Kevin Leyton-Brown
4. Frank Hutter
5. Haejun Lee
6. Yoav Levine
7. Amnon Shashua
8. Ori Ram

In Computer Vision and Pattern Recognition

1. Hamed Zamani
2. Haejun Lee
3. Yoav Levine
4. Jiecao Chen
5. Sebastian Hofstätter
6. Amnon Shashua
7. Karthik Raman
8. Ori Ram

In Retrieval-Augmented Generation

1. Christopher D. Manning
2. Karthik Raman
3. Hamed Zamani
4. Jiecao Chen
5. Ashwin Paranjape
6. Ori Ram
7. Sebastian Hofstätter

In Natural Language Processing

1. Hamed Zamani
2. Sebastian Hofstätter
3. Haejun Lee
4. Ori Ram
5. Amnon Shashua
6. Yoav Levine
7. Ashwin Paranjape
8. Jiecao Chen

Citations in the corpus

• Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling (2022, background): “R2-D2 (Fajcik et al., 2021)), or creating multiple training loops to update the indexed documents multiple times (f.”
• You Only Need One Model for Open-domain Question Answering (2021, background): “UnitedQA (Cheng et al., 2021) and R2D2 (Fajcik et al., 2021) combine re-sults from an ensemble of extractive and generative readers, whereas PAQ (Lewis et al., 2021) directly retrieves answers with an FiD fallback.”
• Rethinking the Objectives of Extractive Question Answering (2020, unknown): “We further hypothesize that having diverse answers may be especially beneficial towards answer reranking step commonly used in QA (Fajcik et al., 2021; Iyer et al., 2021).”
• REPLUG: Retrieval-Augmented Black-Box Language Models (2023, unknown): “The second group of models for comparison includes retrieval-augmented language models such as RETRO (Borgeaud et al., 2021), R2-D2 (Fajcik et al., 2021), and Atlas (Izacard et al., 2022b).”
• In-Context Retrieval-Augmented Language Models (2023, result): “In contrast to most prior work on ODQA (e.g., Izacard and Grave 2021; Fajcik et al. 2021; Izacard et al. 2022b; Levine et al. 2022b), our “reader” (i.”
• LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (2024, unknown): “…(Chen et al., 2023b) 57.6 HGN (Fang et al., 2019) 59.7 PathRetriever (Asai et al., 2019) 60.0 HopRetrieve (Li et al., 2021) 62.1 MDR (Xiong et al., 2020b) 62.3 HopRetrieve-plus (Li et al., 2021) 66.5 AISO (Zhu et al., 2021) 68.1 COS (Ma et al., 2023) 68.2 No Fine-tuning RAG DSP (Khattab et al.,…”