Neural Logic Machines
By Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, Denny Zhou. 2019
302 citations recorded by the source corpus.
We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning. NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifiers. After being trained on small-scale tasks (such as sorting short arrays), NLMs can recover lifted rules, and generalize to large-scale tasks (such as sorting longer arrays). In our experiments, NLMs achieve perfect generalization in a number of tasks, from relational reasoning tasks on the family tree and general graphs, to decision making tasks including sorting arrays, finding shortest paths, and playing the blocks world. Most of these tasks are hard to accomplish for neural networks or inductive logic programming alone.
Subfields: Artificial Intelligence, Natural Language Processing, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Luke S. Zettlemoyer
- Quoc V. Le
- Yoshua Bengio
- Jianfeng Gao
- Graham Neubig
- Noah A. Smith
In Natural Language Processing
- Mohit Bansal
- Luke S. Zettlemoyer
- Noah A. Smith
- Percy Liang
- Graham Neubig
- Yejin Choi
- Christopher D. Manning
- Jonathan Berant
In Knowledge Graphs & Representation
- A.S. d'Avila Garcez
- Luciano Serafini
- Michael Spranger
- Marco Gori
- L. C. Lamb
- Quoc V. Le
- Son Dinh Tran
- Sebastijan Dumancic
Citations in the corpus
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “Recent work along in this direction has included a number of architectural innovations (Cai et al., 2017; Dong et al., 2019; Yan et al., 2020), as well as the use of large language models (Chen et al.”
- Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, background): “Some works seek to use distributed representations to augment the rule-based backward chaining (Weber et al., 2019; Dong et al., 2018).”
- Neural Execution Engines: Learning to Execute Subroutines (2020, background): “Work in neural program synthesis [16, 18, 5, 1, 6]—which uses neural networks with the goal of generating and finding a “correct” program such that it will generalize beyond the training distribution—has also employed strong supervision in the form of execution traces [20, 21, 2].”
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models (2022, methodology): “In this respect, it shares the same spirit as the methods in (Graves, 2016; Dehghani et al., 2019; Dong et al., 2019; Schwarzschild et al., 2021; Wang et al., 2021; Wei et al., 2022; Wang et al., 2022).”
- Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning (2019, methodology): “Different from [57, 14, 39] where clauses are generated and then softened by neural networks, in Neural Logic Machines (NLM) [13] the relation of predicates is learned by a neural network where input tensors represent facts (predicates of different arities) from a knowledge base and output tensors rep”
- DeepProbLog: Neural Probabilistic Logic Programming (2018, background): “[50] induce programs in a purely neural way and demonstrate favourable generalization; however, they do not induce symbolic programs, but rather express a program through a neural network.”
- Measuring and Narrowing the Compositionality Gap in Language Models (2022, methodology): “Allowing models to use more computation to solve harder problems has been previously explored in both language (Graves, 2016; Seo et al., 2017; Dong et al., 2019; Schwartz et al., 2020; Schwarzschild et al., 2021; Nye et al., 2021; Wei et al., 2022b; Zhou et al., 2022) and vision (Bolukbasi et al.,…”
- Neural Learning of One-of-Many Solutions for Combinatorial Problems in Structured Output Spaces (2020, methodology): “Models for Symbolic Reasoning: Our work follows the line of recent research, which proposes neural architectures for implicit symbolic and relational reasoning problems [Santoro et al., 2018, Palm et al., 2018, Wang et al., 2019, Dong et al., 2019].”
- Neural Logic Reinforcement Learning (2019, methodology): “t al., 2018), the learned policy is not fully explainable and the rules expression is limited, different from the interpretable logic-represented policies learned in ours using DILP. A parallel work (Dong et al., 2019) use neural networks to approximate the first-order logic deduction, achieving good”
- Exploring End-to-End Differentiable Natural Logic Modeling (2020, methodology): “For example, the work in (Rockt¨aschel and Riedel, 2017; Weber et al., 2019; Minervini et al., 2020) proposes a differentiable backward-chaining algorithm, and Dong et al. (2019) adopt probabilistic tensor representations for logic predicates and mimic the forward-chaining proof.”
- Conversational Neuro-Symbolic Commonsense Reasoning (2020, background): “They have shown promise in different areas of logical reasoning ranging from classical logic to propositional logic, probabilistic logic, abductive logic, and inductive logic [30–35].”
- Neurosymbolic AI: the 3rd wave (2020, unknown): “l-symbolic integration, so-called embedding techniques seek to transform symbolic representations into vector spaces where reasoning can take place through matrix computations over distance functions [7, 82, 83, 77, 72, 11, 27, 96, 25, 67]. In such systems, learning of an embedding is carried out us”
- Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020, background): “Recent semantic parsing methods strives to leverage the power of neural networks (Neelakantan et al., 2016; Jia and Liang, 2016; Liang et al., 2017; Yu et al., 2018; Dong et al., 2019a).”
- Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (2022, background): “There are two broad research approaches for reasoning in NLP: (1) reasoning over a formal symbolic language, possibly with neuro-symbolic methods and/or semantic parsing (Saparov & Mitchell, 2022; Kapanipathi et al., 2021; Dong et al., 2019; Rocktäschel & Riedel, 2017), or (2) reasoning directly ove”
- Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension (2020, background): “studied neural symbolic approaches to learn the execution of symbolic operations such as addition and sorting (Graves et al., 2014; Reed & de Freitas, 2016; Cai et al., 2017; Dong et al., 2019).”
- Compositional Generalization via Neural-Symbolic Stack Machines (2020, background): “Some recent work also studies length generalization for other tasks, including relational reasoning [14], multi-task learning [7], and structure inference [31].”
- A large-scale benchmark for few-shot program induction and synthesis (2021, background): “Many of these approaches propose neural architectures inspired by computational modules (Graves et al., 2014; Kurach et al., 2015; Reed & De Freitas, 2015; Joulin & Mikolov, 2015; Graves et al., 2016; Dong et al., 2019; Li et al., 2020), training weights end-to-end.”