Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering
By Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Kumar Rana, Suranga Nanayakkara. Transactions of the Association for Computational Linguistics, 2022
351 citations recorded by the source corpus.
Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA). RAG has only been trained and explored with a Wikipedia-based external knowledge base and is not optimized for use in other specialized domains such as healthcare and news. In this paper, we evaluate the impact of joint training of the retriever and generator components of RAG for the task of domain adaptation in ODQA. We propose RAG-end2end, an extension to RAG that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. In addition, we introduce an auxiliary training signal to inject more domain-specific knowledge. This auxiliary signal forces RAG-end2end to reconstruct a given sentence by accessing the relevant information from the external knowledge base. Our novel contribution is that, unlike RAG, RAG-end2end does joint training of the retriever and generator for the end QA task and domain adaptation. We evaluate our approach with datasets from three domains: COVID-19, News, and Conversations, and achieve significant performance improvements compared to the original RAG model. Our work has been open-sourced through the HuggingFace Transformers library, attesting to our work’s credibility and technical consistency.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/6fcdad7b8d6b60b23bc51859e736c29f913b249a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
Citations in the corpus
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (2024, positive): “…RAG has the potential to substantially mitigate many of the kinds of legal hallucinations that are known to afflict off-the-shelf LLMs (Dahl et al., 2024)—the technique performs well in many general question-answering situations (Guu et al., 2020; Lewis et al., 2020; Siriwardhana et al., 2023).”
- Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, neutral): “RAG-end2end [228] conducts simultaneous training of the retriever (DPR) and the generator (BART) to optimize performance for the end-to-end question-answering task and to facilitate domain adaptation.”