Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents

By Rui Zhang, Honglak Lee, Dragomir R. Radev. 2016

132 citations recorded by the source corpus.

The goal of sentence and document modeling is to accurately represent the meaning of sentences and documents for various Natural Language Processing tasks. In this work, we present Dependency Sensitive Convolutional Neural Networks (DSCNN) as a generalpurpose classification system for both sentences and documents. DSCNN hierarchically builds textual representations by processing pretrained word embeddings via Long ShortTerm Memory networks and subsequently extracting features with convolution operators. Compared with existing recursive neural models with tree structures, DSCNN does not rely on parsers and expensive phrase labeling, and thus is not restricted to sentencelevel tasks. Moreover, unlike other CNNbased models that analyze sentences locally by sliding windows, our system captures both the dependency information within each sentence and relationships across sentences in the same document. Experiment results demonstrate that our approach is achieving state-ofthe-art performance on several tasks, including sentiment analysis, question type classification, and subjectivity classification.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learned in Translation: Contextualized Word Vectors (2017, neutral): “…resource for transfer learning in NLP. [Looks et al., 2017] 89.4 SVM [Van-Tu and Anh-Cuong, 2016] 95.2 TE-LSTM [Huang et al., 2017] 89.6 DSCNN-P [Zhang et al., 2016] 95.6 NSE [Munkhdalai and Yu, 2016a] 89.7 BCN+Char+CoVe (Ours) 95.8 BCN+Char+CoVe (Ours) 90.3 TBCNN [Mou et al., 2015] 96.0 bmLSTM…”
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “The output of the i th word is shown in the following equation: Here, element-wise sum is used to combine the forward and backward pass outputs.”
A Survey of Cross-lingual Word Embedding Models (2017, background): “Zhang et al. (2016b) furthermore map proper nouns to nouns and symbol makers (e.g. “-”, “/”) and interjections to an X tag as it is hard and unnecessary to disambiguate them in a low-resource setting.”
Weighted Neural Bag-of-n-grams Model: New Baselines for Text Classification (2016, methodology): “The first is order/syntax-aware models, such as Convolutional Neural Networks (CNNs) (Kim, 2014; Kalchbrenner et al., 2014; Zhang et al., 2016), Recurrent Neural Networks (RNNs) (Dai and Le, 2015) and Recursive Neural Networks (RecNNs) (Socher et al.”
Deep Learning--based Text Classification (2020, methodology): “(b) The architecture of DSCNN for document modeling [130].”
Deep Learning for Extreme Multi-label Text Classification (2017, methodology): “…state-of-the-art. strong deep learning models in multi-class text classification include the convolutional neural network by [25] (CNN), the recurrent neural network by [27] (RNN), the combination of CNN and RNN by [49], the CNN with attention mechanism by [2, 43] and the Bow-CNN model by [21, 22].”