When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
By Alex Troy Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, Daniel Khashabi. 2022
1,304 citations recorded by the source corpus.
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the difficulty of encoding a wealth of world knowledge in their parameters. This paper aims to understand LMs’ strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments on two open-domain entity-centric QA datasets: PopQA, our new dataset with 14k questions about long-tail entities, and EntityQuestions, a widely used open-domain QA dataset. We find that LMs struggle with less popular factual knowledge, and that retrieval augmentation helps significantly in these cases. Scaling, on the other hand, mainly improves memorization of popular knowledge, and fails to appreciably improve memorization of factual knowledge in the tail. Based on those findings, we devise a new method for retrieval-augmentation that improves performance and reduces inference costs by only retrieving non-parametric memories when necessary.
Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Large Language Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hannaneh Hajishirzi
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Jonathan Berant
- Jianfeng Gao
- Yejin Choi
- Wen-tau Yih
In Retrieval-Augmented Generation
- Hannaneh Hajishirzi
- Akari Asai
- Luke S. Zettlemoyer
- Sewon Min
- Wen-tau Yih
- Jonathan Berant
- Michael Lewis
- Mohit Iyyer
In Large Language Models
- Mohit Bansal
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Jonathan Berant
- Wen-tau Yih
- Michael Lewis
- Akari Asai
- Sewon Min
Citations in the corpus
- Humans or LLMs as the Judge? A Study on Judgement Bias (2024, background): “Your role is to discern which response is superior based on the quality of the answer .”
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2023, methodology): “Hallucination vs Models Table 3 illustrates in Table 4, there is a clear overall trend of an in-411 Table 6: The span-level detection performance for each baseline method across different tasks and different models. provide a response-level detection outcome.”
- Evaluating Very Long-Term Conversational Memory of LLM Agents (2024, background): “…dialogues often feature co-references and missing content ( i.e., anaphora) (Anantha et al., 2021), which further complicate the retrieval process (Mallen et al., 2023; Gao et al., 2023b; Liu et al., 2023b); (2) Challenges arise in reasoning over retrieved documents, especially when the model…”
- Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, neutral): “Mallen et al. [211] used statistical analysis on questions to enable direct answers for high-frequency ones and applied RAG for low-frequency ones.”
- Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, neutral): “, 2021) Generation-Time Search API QA Adaptive-Retrieval (Mallen et al., 2023) Generation-Time Wikipedia QA ReACT (Yao et al.”
- Group-in-Group Policy Optimization for LLM Agent Training (2025, neutral): “Next, we conducted an ablation study, comparing GiGPO w/o std ( F norm = 1 ), GiGPO w/ std ( F norm = std), GiGPO w/o A S (without step relative advantages), and GiGPO w/o A E (without episode relative advantages) to evaluate the impact of each component on performance.”
- Large Language Models for Information Retrieval: A Survey (2023, neutral): “More severely, researchers [229, 230] show that some modules in IR systems such as retriever and reranker, especially those based on neural models, may prefer LLM-generated documents, since their topics are more consistent and the perplexity of them are lower compared with human-written documents.”
- Tool learning with large language models: a survey (2024, positive): “The definition of a tool is notably broad within the context of augmented LLMs. Mi-alon et al. (2023) [45] articulate a tool as “ the exter-nal employment of an unattached or manipulable attached environmental object to alter more efficiently the form, position, or condition of another object.”
- Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024, background): “Moreover, a fair portion of knowledge in the wild is long-tailed, making it challenging to recall during deployment [30, 38].”
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (2023, methodology): “This can be partially explained by the ability of models to combine retrieval and their parametric knowledge (Talmor et al., 2020; Zhong et al., 2023; Cohen et al., 2023).”
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “• Long-tail Knowledge Within the vast knowledge landscape that LLMs draw upon, a notable challenge emerges in the form of long-tail knowledge utilization (Kandpal et al., 2023; Mallen et al., 2023).”
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023, background): “This is in agreement with Kandpal et al. (2022) and Mallen et al. (2023) which show that short question answering (QA) accuracy is highly correlated with the entity frequencies in the pretraining data.”
- RAGAs: Automated Evaluation of Retrieval Augmented Generation (2023, background): “Second, even the largest models struggle to memorise knowledge that is only rarely mentioned in the training corpus (Kandpal et al., 2022; Mallen et al., 2023).”
- Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024, methodology): “We refer to our framework as Adaptive Retrieval-Augmented Generation (Adaptive-RAG).”
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023, background): “Short-form generations tasks include two open-domain question answering (QA) datasets, PopQA (Mallen et al., 2023) and TriviaQA-unfiltered (Joshi et al., 2017), where systems need to answer arbitrary questions about factual knowledge.”
- Information Asymmetry across Language Varieties: A Case Study on Cantonese-Mandarin and Bavarian-German QA (2026, unknown): “With LLMs, information is typically accessed from model-internal parametric knowledge (Mallen et al., 2023) acquired during pre-training, or from externally provided knowledge in the case of retrieval-augmented LLMs. LLMs degrade markedly when not prompted in English (Zhang et al., 2023).”