NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language

By Leon Weber, Pasquale Minervini, Jannes Munchmeyer, Ulf Leser, Tim Rocktaschel. 2019

107 citations recorded by the source corpus.

Rule-based models are attractive for various tasks because they inherently lead to interpretable and explainable decisions and can easily incorporate prior knowledge. However, such systems are difficult to apply to problems involving natural language, due to its large linguistic variability. In contrast, neural models can cope very well with ambiguity by learning distributed representations of words and their composition from data, but lead to models that are difficult to interpret. In this paper, we describe a model combining neural networks with logic programming in a novel manner for solving multi-hop reasoning tasks over natural language. Specifically, we propose to use an Prolog prover which we extend to utilize a similarity function over pretrained sentence encoders. We fine-tune the representations for the similarity function via backpropagation. This leads to a system that can apply rule-based reasoning to natural language, and induce domain-specific natural language rules from training data. We evaluate the proposed system on two different question answering tasks, showing that it outperforms two baselines – BiDAF (Seo et al., 2016a) and FastQA( Weissenborn et al., 2017) on a subset of the WikiHop corpus and achieves competitive results on the MedHop data set (Welbl et al., 2017).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, background): “Some works seek to use distributed representations to augment the rule-based backward chaining (Weber et al., 2019; Dong et al., 2018).”
Flexible Generation of Natural Language Deductions (2021, background): “Systems like NLProlog (Weber et al., 2019) attempt to explicitly ground reasoning in logic, but this process still heavily relies on latent representations; in contrast, by grounding reasoning directly in natural language, a system based on natural deduction operations like ours gains inherent faith”
Transformers as Soft Reasoners over Language (2020, methodology): “, [Weber et al., 2019; Minervini et al., 2019; Minervini et al., 2018].”
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language (2020, methodology): “menting a reasoning algorithm with neural components (Weber et al., 2019; Rocktäschel and Riedel, 2017), or SAT solving (Selsam et al.”
PRover: Proof Generation for Interpretable Reasoning over Rules (2020, neutral): “Our approach is conceptually similar to a body of work on Neural Theorem Proving (Weber et al., 2019) that has focused on developing theorem provers by combining reasoning from symbolic techniques with the possibility of differentiable learning from neural networks.”
Learning to Annotate: Modularizing Data Augmentation for Text Classifiers with Natural Language Explanations (2019, methodology): “2) for NLPROLOG.”
Learning Inference Rules with Neural TP-Reasoner (2020, methodology): “Following Yang et al. (2017), we randomly split the datasets into facts, train, test data with ratio 6:2:1.”
Conversational Neuro-Symbolic Commonsense Reasoning (2020, methodology): “The use of Prolog for performing multi-hop logical reasoning has been studied in [40,41].”
Differentiable Reasoning on Large Knowledge Bases and Natural Language (2019, background): “A promising strategy for overcoming these issues consists of combining neural models and symbolic reasoning, given their complementary strengths and weaknesses (d’Avila Garcez et al. 2015; Rocktäschel and Riedel 2017; Yang, Yang, and Cohen 2017; Evans and Grefenstette 2018; Weber et al. 2019).”
Case-Based Abductive Natural Language Inference (2020, background): “The work presented in this paper is related to hybrid neuro-symbolic approaches for inference with natural language (Liu et al., 2020; Minervini et al., 2020; Jiang and Bansal, 2019; Chen et al., 2019; Dua et al., 2019; Xu et al., 2021; Weber et al., 2019).”
Neurosymbolic AI: the 3rd wave (2020, unknown): “However, the value of distributed approximate reasoning using neural networks is only starting to be investigated as in the case of differentiable neural computers [33] and neural theorem proving [97], although early efforts did not prove to be promising in terms of practical efficiency [62, 38, 82]”
Exploring End-to-End Differentiable Natural Logic Modeling (2020, methodology): “…of bringing together the complementary strengths of neural networks and symbolic models, e.g., the research performed in (Garcez et al., 2015; Yang et al., 2017; Rockt¨aschel and Riedel, 2017; Evans and Grefenstette, 2018; Weber et al., 2019; De Raedt et al., 2019; Mao et al., 2019), among others.”
Learning from Explanations with Neural Execution Tree (2019, unknown): “2) for NLPROLOG.”