Dependency-based Convolutional Neural Networks for Sentence Embedding

By Mingbo Ma, Liang Huang, Bowen Zhou, Bing Xiang. 2015

121 citations recorded by the source corpus.

In sentence modeling and classification, convolutional neural network approaches have recently achieved state-of-the-art results, but all such efforts process word vectors sequentially and neglect long-distance dependencies. To exploit both deep learning and linguistic structures, we propose a tree-based convolutional neural network model which exploit various long-distance relationships between words. Our model improves the sequential baselines on all three sentiment and question classification tasks, and achieves the highest published accuracy on TREC.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

A Structured Self-attentive Sentence Embedding (2017, background): “Additional works have also been done in exploiting linguistic structures such as parse and dependence trees to improve sentence representations (Ma et al., 2015; Mou et al., 2015b; Tai et al., 2015).”
A Primer on Neural Network Models for Natural Language Processing (2015, background): “For example, Ma et al (2015) generalize the convolution operation to work over syntactic dependency trees.”
Neural Semantic Role Labeling with Dependency Path Embeddings (2016, background): “Tai et al. (2015) and Ma et al. (2015) learn embeddings of dependency structures representing full sentences, in a sentiment classiﬁ-cation task.”
The Forest Convolutional Network: Compositional Distributional Semantics with a Neural Chart and without Binarization (2015, background): “Recently, Zhu et al. (2015) and Ma et al. (2015) try to intergrate syntactic information by employing parse trees.”
Learning to Compose Words into Sentences with Reinforcement Learning (2016, methodology): “The last approach for constructing sentence representations uses convolutional neural networks to produce the representation in a bottom up manner, either with syntactic information (Ma et al., 2015) or without (Kim, 2014; Kalchbrenner et al., 2014).”
Improved Relation Extraction with Feature-Rich Compositional Embedding Models (2015, methodology): “In order to better utilize the dependency annotations, recently work built their models according to the dependency paths (Ma et al., 2015; Liu et al., 2015), which share similar motivations to the usage of On-path features in our work.”
Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents (2016, background): “First, recursive models depend on well-performing parsers, which can be difficult for many languages or noisy domains (Iyyer et al., 2015; Ma et al., 2015).”