The Forest Convolutional Network: Compositional Distributional Semantics with a Neural Chart and without Binarization

By Phong Le, Willem H. Zuidema. 2015

37 citations recorded by the source corpus.

According to the principle of compositionality, the meaning of a sentence is computed from the meaning of its parts and the way they are syntactically combined. In practice, however, the syntactic structure is computed by automatic parsers which are far-from-perfect and not tuned to the specifics of the task. Current recursive neural network (RNN) approaches for computing sentence meaning therefore run into a number of practical difficulties, including the need to carefully select a parser appropriate for the task, deciding how and to what extent syntactic context modifies the semantic composition function, as well as on how to transform parse trees to conform to the branching settings (typically, binary branching) of the RNN. This paper introduces a new model, the Forest Convolutional Network, that avoids all of these challenges, by taking a parse forest as input, rather than a single tree, and by allowing arbitrary branching factors. We report improvements over the state-of-the-art in sentiment analysis and question classification.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

In Computational Linguistics & Semantics

Citations in the corpus

Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure (2017, neutral): “, 2015), sentiment analysis (e.g. Kalchbrenner et al., 2014; Socher et al., 2013; Le and Zuidema, 2015), semantic role labelling (e.”
A Primer on Neural Network Models for Natural Language Processing (2015, methodology): “Le and Zuidema (2015) propose to perform max pooling over vectors representing the different derivations leading to the same chart item in a chart parser.”
Easy-First Dependency Parsing with Hierarchical Tree LSTMs (2016, methodology): “ approaches that ﬁrst produced a k-best list of parses using a traditional parsing technique, and then scores the trees based on a recursive vector encoding of each node (Le and Zuidema, 2014; Le and Zuidema, 2015; Zhu et al., 2015). Our parser is a greedy, bottom up parser that rely on compositiona”
Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation (2016, background): “Tai et al., (2015) presented tree-structured LSTMs , while Le and Zuidema (2015) further introduced the forest convolutional network for sentence modelling.”
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (2019, methodology): “Our model builds on existing work developing latent tree chart parsers (Socher et al., 2011b; Le and Zuidema, 2015; Yogatama et al., 2017; Mail-lard et al., 2017; Choi et al., 2018).”
Diagnostic Classifiers Revealing how Neural Networks Process Hierarchical Structure (2016, unknown): “A common approach to train and evaluate sentence representations is to have them execute a task that requires semantic knowledge of the sentence such as sentiment analysis, logical inference or question-answering [e.g. Le and Zuidema, 2015a,b, Hermann et al., 2015].”
Visually Grounded Compound PCFGs (2020, methodology): “To realize this desideratum, we could follow the inside algorithm and recursively compose span representations (Le and Zuidema, 2015; Stern et al., 2017; Drozdov et al., 2019), which is, however, time-and memory-inefﬁcient in practice.”
Latent Tree Learning with Differentiable Parsers: Shift-Reduce Parsing and Chart Parsing (2018, methodology): “Hence for fairness, in our experiments we replace the softmax-with-temperature of Maillard et al. (2017) with a softmax followed by a straight-through estimator (Bengio et al.”