Towards Neural Theorem Proving at Scale

By Pasquale Minervini, Matko Bosnjak, Tim Rocktaschel, Sebastian Riedel. 2018

44 citations recorded by the source corpus.

Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest. However, their use is severely limited by their computational complexity, which renders them unusable on real world datasets. We focus on the Neural Theorem Prover (NTP) model proposed by Rockt{\"{a}}schel and Riedel (2017), a continuous relaxation of the Prolog backward chaining algorithm where unification between terms is replaced by the similarity between their embedding representations. For answering a given query, this model needs to consider all possible proof paths, and then aggregate results - this quickly becomes infeasible even for small Knowledge Bases (KBs). We observe that we can accurately approximate the inference process in this model by considering only proof paths associated with the highest proof scores. This enables inference and learning on previously impracticable KBs.

Subfields: Artificial Intelligence, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Prompting & Reasoning

Citations in the corpus

Transformers as Soft Reasoners over Language (2020, methodology): “Several researchers have developed methods for Neural Theorem Proving (NTP), combining symbolic and neural methods to reason step-wise over language-derived structures, e.g., [Weber et al. , 2019; Minervini et al. , 2019; Minervini et al. , 2018].”
Differentiable Reasoning on Large Knowledge Bases and Natural Language (2019, background): “Among such systems, NTPs (Rockt¨aschel and Riedel 2017; Minervini et al. 2018) are end-to-end differentiable deductive reasoners based on Prolog’s backward chaining algorithm, where discrete uniﬁcation between atoms is replaced by a differentiable operator computing the similarities between their…”
DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs (2019, methodology): “Other methods like NTPs [25, 30] and MINERVA [8], rely on some type of learned embeddings at training and test time.”