BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

By Nora Kassner, Oyvind Tafjord, Hinrich Schütze, Peter Clark. 2021

74 citations recorded by the source corpus.

Although pretrained language models (PTLMs) contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed, even after specialized training. As a result, it can be hard to identify what the model actually “believes” about the world, making it susceptible to inconsistent behavior and simple errors. Our goal is to reduce these problems. Our approach is to embed a PTLM in a broader system that also includes an evolving, symbolic memory of beliefs – a BeliefBank – that records but then may modify the raw PTLM answers. We describe two mechanisms to improve belief consistency in the overall system. First, a reasoning component – a weighted MaxSAT solver – revises beliefs that significantly clash with others. Second, a feedback component issues future queries to the PTLM using known beliefs as context. We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system, improving both the accuracy and consistency of its answers over time. This is significant as it is a first step towards PTLM-based architectures with a systematic notion of belief, enabling them to construct a more coherent picture of the world, and improve over time without model retraining.

Subfields: Artificial Intelligence, Large Language Models, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Knowledge Graphs & Representation

Citations in the corpus

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, background): “Kassner et al. (2021) update model beliefs by adding in relevant information to the input at test time (to improve consistency under entailment).”
Memory-Based Model Editing at Scale (2022, methodology): “In this vein, SERAC is related to the BeliefBank system (Kassner et al., 2021), which, while primarily intended to improve model consistency, enables editability of some pre-trained models using an external memory, rather than parameter updates.”
Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions (2022, background): “, 2020) and common-sense constraints (Kassner et al., 2021b).”
A Review on Language Models as Knowledge Bases (2022, background): “On the other hand, Kassner et al. (2021) use a feedback mechanism that issues relevant information from a symbolic memory of beliefs as input to the system during test-time in order to improve consistency under entailment.”
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations (2022, background): “Other works explore how to train a model that simulates the symbolic reasoning process, such as logical transformation (Bostrom et al., 2021) and consistent generation of beliefs (Kassner et al., 2021; Dalvi et al., 2022).”
Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback (2021, unknown): “There has been some recent work to ensure consistency of beliefs of a model (Kassner et al., 2021), and more effort is required in this direction to apply to more complex settings like ours.”
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (2023, methodology): “Prior work employs various heuristics, such as negation injection (Niu & Bansal, 2018; Kassner et al., 2021; Gubelmann & Handschuh, 2022) and entity substitution (Longpre et al., 2021; Zhou et al., 2023), and finds that language models (both large and small) tend to be stubborn and cling to their…”