Recursive Neural Networks for Learning Logical Semantics

By Samuel R. Bowman, Christopher Potts, Christopher D. Manning. 2014

33 citations recorded by the source corpus.

Supervised recursive neural network models (RNNs) for sentence meaning have been successful in an array of sophisticated language tasks, but it remains an open question whether they can learn compositional semantic grammars that support logical deduction. We address this question directly by for the first time evaluating whether each of two classes of neural model — plain RNNs and recursive neural tensor networks (RNTNs) — can correctly learn relationships such as entailment and contradiction between pairs of sentences, where we have generated controlled data sets of sentences from a logical grammar. Our first experiment evaluates whether these models can learn the basic algebra of logical relations involved. Our second and third experiments extend this evaluation to complex recursive structures and sentences involving quantification. We find that the plain RNN achieves only mixed results on all three experiments, whereas the stronger RNTN model generalizes well in every setting and appears capable of learning suitable representations for natural language logical inference.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Learned in Translation: Contextualized Word Vectors (2017, methodology): “Recent work in NLP has continued in this direction by using pretrained word representations to improve models for entailment [Bowman et al., 2014], sentiment analysis [Socher et al.”
Learning to Execute (2014, methodology): “There has been related research that used Tree Neural Network (sometimes known as Recursive Neural Networks) to evaluate symbolic mathematical expressions and logical formulas (Zaremba et al., 2014a; Bowman et al., 2014; Bowman, 2013), which is close in spirit to our work.”
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (2015, methodology): “For instance, recursive neural networks have been used for parsing [11], sentiment analysis [2], paraphrase detection [11] and question answering [12] and logical inference [13], among other tasks.”
When Are Tree Structures Necessary for Deep Learning of Representations? (2015, background): “various NLP tasks, among them entailment (Bowman, 2013; Bowman et al., 2014), sentiment analysis (Socher et al.”
Compositional Vector Space Models for Knowledge Base Completion (2015, background): “, 2014) and natural language logical semantics (Bowman et al., 2014).”