Learning Knowledge Base Inference with Neural Theorem Provers

By Tim Rocktaschel, Sebastian Riedel. 2016

61 citations recorded by the source corpus.

In this paper we present a proof-of-concept implementation of Neural Theorem Provers (NTPs), end-to-end differentiable counterparts of discrete theorem provers that perform first-order inference on vector representations of symbols using function-free, possibly parameterized, rules. As such, NTPs follow a long tradition of neural-symbolic approaches to automated knowledge base inference, but differ in that they are differentiable with respect to representations of symbols in a knowledge base and can thus learn representations of predicates, constants, as well as rules of predefined structure. Furthermore, they still allow us to incorporate domainknowledge provided as rules. The NTP presented here is realized via a differentiable version of the backward chaining algorithm. It operates on substitution representations and is able to learn complex logical dependencies from training facts of small knowledge bases.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning (2019, methodology): “In NTP, a neural system is constructed recursively for backward chaining and uniﬁcation where AND and OR operators are represented as networks.”
Integrating Learning and Reasoning with Deep Logic Models (2019, background): “[6]. Blendingsymbolicandsub-symbolicapproachesisoneofthemostchallenging open problem in AI and, recently, a lot of works, often referred as neurosymbolicapproaches[10],havebeenproposedbyseveralauthors[6,14,22,27]. Inthispaper,wepresentDeepLogicModels(DLMs),auniﬁedframeworkto integratelogicalreasonin”
Learning Explanatory Rules from Noisy Data (2017, background): “9 “Learning Knowledge Base Inference with Neural Theorem Provers” Rocktäschel and Riedel (2016) describe a differentiable neural net implementation of a backward chaining inference procedure.”
Neurosymbolic AI: the 3rd wave (2020, unknown): “Related work seeking to explore the advantages of distributed representations of logic include [11], which is based on stochastic logic programs, [26, 100, 24], with a focus on inductive programming, and [70], based on differentiable theorem proving.”
Lifted Rule Injection for Relation Embeddings (2016, background): “implication rules, is in the focus of current research (Rocktäschel et al., 2014; Wang et al., 2014; Bowman et al., 2015; Wang et al., 2015; Vendrov et al., 2016; Hu et al., 2016; Rocktäschel and Riedel, 2016; Cohen, 2016).”
Deep Network Guided Proof Search (2017, methodology): “A fully diﬀerential theorem prover was tested on a few toy problems by Rocktäschel & Riedel (2016).”
Lifted Relational Neural Networks: Efficient Learning of Latent Relational Structures (2018, background): “Recently 3 two other frameworks for combining logic programming and neural networks have been introduced (Rockt¨aschel & Riedel, 2016; Cohen, 2016).”
Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks (2016, methodology): “So far this work has mostly focused on small or synthetic data (Grefenstette, 2013; Bowman et al., 2014; Rocktäschel and Riedel, 2016).”
TensorLog: Deep Learning Meets Probabilistic DBs (2017, background): “More recently (Rockt¨aschel & Riedel, 2016) have proposed a diﬀerentiable theorem prover, in which a proof for an example is unrolled into a network.”