Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization

By Mathijs Mul, Willem H. Zuidema. 2019

18 citations recorded by the source corpus.

Can neural nets learn logic? We approach this classic question with current methods, and demonstrate that recurrent neural networks can learn to recognize first order logical entailment relations between expressions. We define an artificial language in first-order predicate logic, generate a large dataset of sample 'sentences', and use an automatic theorem prover to infer the relation between random pairs of such sentences. We describe a Siamese neural architecture trained to predict the logical relation, and experiment with recurrent and recursive networks. Siamese Recurrent Networks are surprisingly successful at the entailment recognition task, reaching near perfect performance on novel sentences (consisting of known words), and even outperforming recursive networks. We report a series of experiments to test the ability of the models to perform compositional generalization. In particular, we study how they deal with sentences of unseen length, and sentences containing unseen words. We show that set-ups using LSTMs and GRUs obtain high scores on these tests, demonstrating a form of compositionality.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Elia Bruni
2. Verna Dankers
3. Dieuwke Hupkes
4. Qian Liu
5. Dongmei Zhang
6. Mathijs Mul
7. Bin Zhou
8. Yan Gao

In Information Systems

1. Dongmei Zhang
2. Zeqi Lin
3. Elia Bruni
4. Jian-Guang Lou
5. Wenhu Chen
6. Peter Alexander Jansen

In Natural Language Processing

1. Elia Bruni
2. Verna Dankers
3. Dieuwke Hupkes
4. Qian Liu
5. Dongmei Zhang
6. Mathijs Mul
7. Bin Zhou
8. Yan Gao

Citations in the corpus

• Compositionality Decomposed: How do Neural Networks Generalise? (2019, background): “…the vast number of different tests that exist for compositionality [ Lake and Baroni, 2018; Hupkes et al. , 2018; Johnson et al. , 2017; Bahdanau et al. , 2018; Saxton et al. , 2019; Loula et al. , 2018; Dess`ı and Baroni, 2019; Liˇska et al. , 2018; Bowman et al. , 2015; Mul and Zuidema, 2019 ] .”
• Compositional Generalization by Learning Analytical Expressions (2020, background): “Under the topic of logical inference, previous works devoted to testing the ability of neural networks on inferring logical relations between pairs of artificial language utterances [6, 27].”