Reasoning Over Virtual Knowledge Bases With Open Predicate Relations

By Haitian Sun, Pat Verga, Bhuwan Dhingra, Ruslan Salakhutdinov, William W. Cohen. 2021

27 citations recorded by the source corpus.

We present the Open Predicate Query Language (OPQL); a method for constructing a virtual KB (VKB) trained entirely from text. Large Knowledge Bases (KBs) are indispensable for a wide-range of industry applications such as question answering and recommendation. Typically, KBs encode world knowledge in a structured, readily accessible form derived from laborious human annotation efforts. Unfortunately, while they are extremely high precision, KBs are inevitably highly incomplete and automated methods for enriching them are far too inaccurate. Instead, OPQL constructs a VKB by encoding and indexing a set of relation mentions in a way that naturally enables reasoning and can be trained without any structured supervision. We demonstrate that OPQL outperforms prior VKB methods on two different KB reasoning tasks and, additionally, can be used as an external memory integrated into a language model (OPQL-LM) leading to improvements on two open-domain question answering tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Minlie Huang
3. Xipeng Qiu
4. Chenguang Zhu
5. Yang Liu
6. Maosong Sun
7. Jie Tang
8. Ji-rong Wen

Citations in the corpus

• Pre-Trained Models: Past, Present and Future (2021, background): “…entities appear in contexts, they replace entity tokens ’ embedding in an internal Transformer layer with the embedding from outer memory networks (Dhingra et al., 2020; Sun et al., 2021). maintain a virtual knowledge from scratch, and propose a differentiable reasoning training objective over it.”
• A Unified Encoder-Decoder Framework with Entity Memory (2022, methodology): “To better encode relational knowledge, OPQL (Sun et al., 2021) learned latent relational representations for arbitrary entity pairs.”
• A Review on Language Models as Knowledge Bases (2022, methodology): “Sun et al. (2021) improve the idea of a VKB and introduces an additional strategy to learn entity and information through OPQL , utilizing key-value memory to learn relationships between entities and relations. de Jong et al. (2021) take the idea of a VKB and insert it into TOME as non-parametric…”
• Relation-Guided Pre-Training for Open-Domain Question Answering (2021, unknown): “Sun et al. (2021) extend this work by learning a virtual knowledge base by inferring the relation between two co-occurring entity pairs.”