A Review on Language Models as Knowledge Bases

By Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab, Marjan Ghazvininejad. 2022

217 citations recorded by the source corpus.

Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs trained on a sufficiently large (web) corpus will encode a significant amount of knowledge implicitly in its parameters. The resulting LM can be probed for different kinds of knowledge and thus acting as a KB. This has a major advantage over traditional KBs in that this method requires no human supervision. In this paper, we present a set of aspects that we deem a LM should have to fully act as a KB, and review the recent literature with respect to those aspects.

Subfields: Artificial Intelligence, Management Science and Operations Research, Large Language Models, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Large Language Models

In Knowledge Graphs & Representation

Citations in the corpus

Mass-Editing Memory in a Transformer (2022, background): “, 2020), and it remains difficult to catalog, add, or update knowledge (AlKhamissi et al., 2022).”
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models (2022, background): “, on QA tasks) [19, 20, 21], while also allowing the models to encode significant amounts of world or factual knowledge [22, 23, 24].”