Editing Factual Knowledge in Language Models

By Nicola De Cao, Wilker Aziz, Ivan Titov. 2021

715 citations recorded by the source corpus.

The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix ‘bugs’ or unexpected predictions without the need for expensive re-training or fine-tuning. Besides being computationally efficient, KnowledgeEditordoes not require any modifications in LM pre-training (e.g., the use of meta-learning). In our approach, we train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the trained hyper-network is then used to predict the weight update at test time. We show KnowledgeEditor’s efficacy with two popular architectures and knowledge-intensive tasks: i) a BERT model fine-tuned for fact-checking, and ii) a sequence-to-sequence BART model for question answering. With our method, changing a prediction on the specific wording of a query tends to result in a consistent change in predictions also for its paraphrases. We show that this can be further encouraged by exploiting (e.g., automatically-generated) paraphrases during training. Interestingly, our hyper-network can be regarded as a ‘probe’ revealing which components need to be changed to manipulate factual knowledge; our analysis shows that the updates tend to be concentrated on a small subset of components. Source code available at https://github.com/nicola-decao/KnowledgeEditor

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, methodology): “In a similar vein, De Cao et al. (2021); Mitchell et al. (2022) systematically edit model parameters by adding KBs to LMs trained without them, in order to revise facts memorized by the model.”
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022, methodology): “Some recent methods have also attempted to adapt knowledge through editing and tuning of language model variants (De Cao et al., 2021; Mitchell et al., 2022).”
Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “A related line of work has explored editing neural predictions after training given a dataset of revised input and output pairs (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021).”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “An alternative, recently explored approach is “model editing” - updating the model itself by modifying its parameters to fix incorrect answers (Mitchell et al., 2021; De Cao et al., 2021; Hase et al., 2021).”
Learning by Distilling Context (2022, neutral): “This is in contrast to prior works (Mitchell et al., 2021; De Cao et al., 2021; Meng et al., 2022) on fact editing, which instead perform a constrained optimization procedure that directly learns an edit to the model parameters, corresponding to the factual knowledge update.”
Locating and Editing Factual Associations in GPT (2022, methodology): “De Cao et al. (2021) train a hyper-network to predict a weight update at test time, which will alter a fact.”
GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “Researchers can also modify the model architecture and weights, to validate the proposed algorithms to improve LLMs Zhu et al. (2020); Cao et al. (2021); Hase et al. (2021); Mitchell et al. (2022).”
Mass-Editing Memory in a Transformer (2022, methodology): “The approaches include constrained fine-tuning (Zhu et al., 2020), hypernetwork knowledge editing (De Cao et al., 2021; Hase et al., 2021; Mitchell et al., 2021; 2022), and rank-one model editing (Meng et al., 2022).”
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, methodology): “We propose a Sequential, Local, and Generalizing belief update objective (SLAG) that substantially improves the performance of the K NOWLEDGE E DITOR method from De Cao et al. (2021).”
Fast Model Editing at Scale (2021, methodology): “We conduct an additional experiment editing the BERT-base and BART-base models ﬁne-tuned by Cao et al. (2021) on the FEVER fact-checking and zsRE question-answering tasks, respectively, and our Wikitext editing task, editing a smaller distilGPT-2 model (Wolf et al., 2019) ﬁne-tuned on Wikitext2…”
Towards Continual Knowledge Learning of Language Models (2021, methodology): “(1) While regularization methods (Kirkpatrick et al., 2017) require identifying important parameters used for previous tasks, exactly how and where the knowledge is stored in the parameters of an LM is currently extremely difficult to identify and localize (Vig et al., 2020; De Cao et al., 2021).”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “A related approach is to disentangle the acquisition of up-to-date knowledge from the language learning itself, and enable direct editing of that knowledge within the model (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021).”
Memory-Based Model Editing at Scale (2022, methodology): “Similarly to past work (De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021 (2) The counterfactual model h ψ considers an edit z e and a corresponding example ( x in , y in ) ∼ I ( z e ; D e ) , and is trained to minimize the negative log likelihood of y in given z e and x in on average…”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “Knowledge editing De Cao et al. (2021); Sinitsin et al. (2020) has garnered rising attention from researchers, which aims to rectify model behavior by incorporating additional knowledge.”
Locating and Editing Factual Knowledge in GPT (2022, methodology): “Research has been done for masked models (Petroni et al., 2019; Jiang et al., 2020; Elazar et al., 2021a; Geva et al., 2021; Dai et al., 2021; De Cao et al., 2021), but GPT’s architectural differences (e.g., unidirectional attention, generation capabilities) provide an opportunity for new insights.”
Entity Cloze By Date: What LMs Know About Unseen Entities (2022, background): “Since the entity knowledge in our dataset is time-indexed, this is suitable for evaluating knowledge editing approaches (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021; Mitchell et al., 2021; Meng et al., 2022) and also continual knowledge learning approaches (Jang et al., 2021).”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, positive): “To mitigate these issues in LLMs with minimal computational overhead, the concept of model editing has been introduced (Sinitsin et al., 2020; De Cao et al., 2021).”
Can Large Language Models Be an Alternative to Human Evaluations? (2023, background): “First, LLM may possess incorrect factual knowledge (Cao et al., 2021), so it is not suitable to use them in tasks that involve factual knowledge.”
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions (2023, background): “…words, when injecting a small number of edits, these techniques successfully inject the ed-its into language models and have the edited model recall them at inference time, which corroborates previous findings (Zhu et al., 2021; De Cao et al., 2021; Meng et al., 2022a; Mitchell et al., 2022a,b).”