Modifying Memories in Transformer Models

By Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, Sanjiv Kumar. 2020

278 citations recorded by the source corpus.

Large Transformer models have achieved impressive performance in many natural language tasks. In particular, Transformer based language models have been shown to have great capabilities in encoding factual knowledge in their vast amount of parameters. While the tasks of improving the memorization and generalization of Transformers have been widely studied, it is not well known how to make transformers forget specific old facts and memorize new ones. In this paper, we propose a new task of \emph{explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts}. This task is useful in many scenarios, such as updating stale knowledge, protecting privacy, and eliminating unintended biases stored in the models. We benchmarked several approaches that provide natural baseline performances on this task. This leads to the discovery of key components of a Transformer model that are especially effective for knowledge modifications. The work also provides insights into the role that different training phases (such as pretraining and fine-tuning) play towards memorization and knowledge modification.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “Finetuning only on the newest data ( 2019 ) is suboptimal as the model forgets facts about the past (Figure 5), which was also observed by Zhu et al. (2020).”
Locating and Editing Factual Associations in GPT (2022, methodology): “Constrained Fine-Tuning (FT+L) (Zhu et al., 2020) additionally imposes a parameter-spaceL∞ norm constraint on weight changes.”
Mass-Editing Memory in a Transformer (2022, methodology): “Inspired by the observation that robustness is improved when parameter change magnitudes are minimized (Zhu et al., 2020), we spread updates evenly over the range of mediating layers R .”
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, methodology): “We build on work on editing models after training, which is an exciting recent direction of research with many potentially valuable use cases (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021; Mitchell et al., 2021).”
Fast Model Editing at Scale (2021, methodology): “…strategies for model editing exist, including modiﬁcations of standard ﬁne-tuning intended to enforce locality by reducing distance traveled in parameter space (Zhu et al., 2020) or even ﬁnd the min-L2 2021)) learn to edit a base model through meta-learning (Finn et al., 2017; Ha et al., 2017).”
Editing Factual Knowledge in Language Models (2021, methodology): “Fine-tuning with Zhu et al.’s (2020) method does not affect performance for FC much, which is not surprising since standard ﬁne-tuning already gets almost perfect scores.”
Towards Continual Knowledge Learning of Language Models (2021, methodology): “A.3 METHODS OF INTEGRATING WORLD KNOWLEDGE WITH LANGUAGE MODELS Explicit Methods Facts-as-Experts (Verga et al., 2021) store representations of entities in the form of key-value pairs into external memory that can be modified during inference time.”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “A related approach is to disentangle the acquisition of up-to-date knowledge from the language learning itself, and enable direct editing of that knowledge within the model (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021).”
Memory-Based Model Editing at Scale (2022, methodology): “…base model ’s behavior for only a small region of the domain, without damaging model performance on other inputs of interest (Sinitsin et al., 2020; Zhu et al., 2020; Sotoudeh & Thakur, 2019; De Cao et al., 2021; Dai et al., 2021; Mitchell et al., 2021; Hase et al., 2021; Meng et al., 2022 Edit…”
Exploring AI Ethics of ChatGPT: A Diagnostic Analysis (2023, background): “We kindly mention that some existing works of weight editing [70, 71, 72] could partially address the problem, but impractically.”
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023, methodology): “In §2.1, we aim to extend LLM knowledge bases through finetuning, as in Zhu et al. (2020).”
A Review on Language Models as Knowledge Bases (2022, methodology): “This method can achieve high performance on the modified facts D, but can significantly degrade the model’s performance on the unmodified facts (Zhu et al., 2020).”
Moving the Eiffel Tower to ROME: Tracing and Editing Facts in GPT (2021, background): “Broadly, there are fine-tuning meth-456 ods (Zhu et al., 2020), hypernetwork-based meth-457 ods (Mitchell et al., 2021; De Cao et al., 2021), as well as a neuron interpretability approach (Dai et al., 2021).”
Editing Large Language Models: Problems, Methods, and Opportunities (2023, unknown): “, 2020), which improves model adaptability across tasks and domains, has shown effectiveness in model editing in PLMs (Zhu et al., 2020).”
Locating and Editing Factual Knowledge in GPT (2022, methodology): “For constrained ﬁne-tuning (FT+L), we draw from Zhu et al. (2020) by adding an L ∞ norm constraint: (cid:107) θ G − θ G (cid:48) (cid:107) ∞ ≤ (cid:15) .”
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions (2023, methodology): “We follow Zhu et al. (2021) and use a norm constraint on weight changes with a coefficient 5 × 10 − 5 in our implementation.”
Entity Cloze By Date: What LMs Know About Unseen Entities (2022, background): “Since the entity knowledge in our dataset is time-indexed, this is suitable for evaluating knowledge editing approaches (Sinitsin et al., 2020; Zhu et al., 2020; De Cao et al., 2021; Mitchell et al., 2021; Meng et al., 2022) and also continual knowledge learning approaches (Jang et al., 2021).”
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors (2022, unknown): “Recent approaches all study transformer architectures due to their popularity and high training cost [48, 51].”