Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs

By Peter Hase, Mona T. Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srini Iyer. 2021

92 citations recorded by the source corpus.

Do language models have beliefs about the world? Dennett (1995) famously argues that even thermostats have beliefs, on the view that a belief is simply an informational state decoupled from any motivational state. In this paper, we discuss approaches to detecting when models have beliefs about the world, and we improve on methods for updating model beliefs to be more truthful, with a focus on methods based on learned optimizers or hypernetworks. Our main contributions include: (1) new metrics for evaluating belief-updating methods that focus on the logical consistency of beliefs, (2) a training objective for Sequential, Local, and Generalizing model updates (SLAG) that improves the performance of learned optimizers, and (3) the introduction of the belief graph, which is a new form of interface with language models that shows the interdependencies between model beliefs. Our experiments suggest that models possess belief-like qualities to only a limited extent, but update methods can both fix incorrect model beliefs and greatly improve their consistency. Although off-the-shelf optimizers are surprisingly strong belief-updating baselines, our learned optimizers can outperform them in more difficult settings than have been considered in past work. Code is available at https://github.com/peterbhase/SLAG-Belief-Updating

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “An alternative, recently explored approach is “model editing” - updating the model itself by modifying its parameters to fix incorrect answers (Mitchell et al., 2021; De Cao et al., 2021; Hase et al., 2021).”
Locating and Editing Factual Associations in GPT (2022, background): “Hase et al. (2021) observed that standard model-editing benchmarks underestimate difﬁculty by often testing only proposals that the model previously scored as likely.”
GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “Researchers can also modify the model architecture and weights, to validate the proposed algorithms to improve LLMs Zhu et al. (2020); Cao et al. (2021); Hase et al. (2021); Mitchell et al. (2022).”
Mass-Editing Memory in a Transformer (2022, methodology): “Achieving these goals is challenging, even for a few edits (Hase et al., 2021; Mitchell et al., 2022; Meng et al., 2022).”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, neutral): “Unlearning behavior in language models is similar to model-editing [22, 44], but for rewards rather than datapoints.”
Memory-Based Model Editing at Scale (2022, methodology): “More recently, new classes of methods develop external learned editors that modify finetuning gradients for editing, but do not change the base model that must process edits (De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021).”
Rethinking machine unlearning for large language models (2024, unknown): “In what follows, we elaborate on the assessment of LLM unlearning in terms of unlearning effectiveness, utility preservation, and efficiency.”
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines (2022, background): “…through Intervention Some recent work has explored editing models by computing targeted changes in the model’s parameters (Sinitsin et al., 2020; Hase et al., 2021; Mitchell et al., 2021; Meng et al., 2022), while others have explored natural language feedback (Madaan et al., 2022; Scheurer et…”
Continual Learning of Large Language Models: A Comprehensive Survey (2024, unknown): “For more detailed introduction, please refer to Appendix A.1.”
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors (2022, unknown): “Accessing these data is often unrealistic or impractical, and existing editor performance severely decays the pre-trained model after only a few sequential edits [12, 16].”
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions (2023, methodology): “The evaluation metrics for knowledge-editing techniques often involve verifying the updated answers by querying the editing facts as well as verifying that irrelevant facts are not corrupted (Meng et al., 2022a; Mitchell et al., 2022a; De Cao et al., 2021; Zhu et al., 2021; Hase et al., 2023).”
A Review on Language Models as Knowledge Bases (2022, methodology): “Hase et al. (2021) measure consistency under entailment, including contrapositives, after updating beliefs of a LM. Specifically, they adapt the LeapOfThought dataset (Talmor et al., 2020b) such that each datapoint has a main fact (e.g.”