Document Modeling with Gated Recurrent Neural Network for Sentiment Classification

By Duyu Tang, Bing Qin, Ting-Wei Liu. 2015

1,495 citations recorded by the source corpus.

Document level sentiment classification remains a challenge: encoding the intrinsic relations between sentences in the semantic meaning of a document. To address this, we introduce a neural network model to learn vector-based document representation in a unified, bottom-up fashion. The model first learns sentence representation with convolutional neural network or long short-term memory. Afterwards, semantics of sentences and their relations are adaptively encoded in document representation with gated recurrent neural network. We conduct document level sentiment classification on four large-scale review datasets from IMDB and Yelp Dataset Challenge. Experimental results show that: (1) our neural model shows superior performances over several state-of-the-art algorithms; (2) gated recurrent neural network dramatically outperforms standard recurrent neural network in document modeling for sentiment classification. 1

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

Deep Pyramid Convolutional Neural Networks for Text Categorization (2017, methodology): “Examples are deep character-level CNNs (Zhang et al., 2015; Conneau et al., 2016), a complex combination of CNNs and recurrent neural networks (RNNs) (Tang et al., 2015), and RNNs in a word-sentence hierarchy (Yang et al., 2016).”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, background): “Although deep neural networks (DNN) have dominated almost every ﬁeld of natural language processing, such as sentiment classiﬁcation (Tang, Qin, and Liu 2015), machine translation (Bahdanau, Cho, and Bengio 2014) and question answering (Zhou et al. 2015), generating high-quality summaries from long…”
Do Not Have Enough Data? Deep Learning to the Rescue! (2019, background): “It encompasses a variety of other tasks such as intent classiﬁcation (Kumar et al. 2019), sentiment analysis (Tang, Qin, and Liu 2015), topic classiﬁcation (Tong and Koller 2001), and relation classiﬁcation (Girid-hara, Mishra, and Modam 2019).”
Hierarchical Attention Networks for Document Classification (2016, methodology): “The neural network based methods are reported in (Tang et al., 2015) and (Zhang et al., 2015).”
Understanding Neural Networks through Representation Erasure (2016, methodology): “Task, Dataset and Training Inspired by recent visualization work from Lei et al. (2016), we focus on the task of document-level aspect rating prediction (Tang et al., 2015a; Tang et al., 2015b).”
Interpretable and Steerable Sequence Learning via Prototypes (2019, methodology): “works (RNNs) and its variants, such as Long Short-Term Memory networks (LSTMs). RNNs have achieved remarkable performance in various sequence modeling applications, e.g., document/text classification [34], machine translation [33] and speech recognition [12]. Despite their superior performance, RNNs”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “Another factor aiding RNN’s suitability for sequence modeling tasks lies in its ability to model variable length of text, including very long sentences, paragraphs and even documents (Tang et al., 2015).”
Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN (2018, background): “For example, hierarchal RNN has been proposed for long document classification [40] and later attention model is also introduced to emphasize important sentences and words [45].”
Attention-based LSTM for Aspect-level Sentiment Classification (2016, methodology): “TC-LSTM added target representations, which was obtained from word vectors, into the input of the LSTM cell unit.”
Towards Open Intent Discovery for Conversational Text (2019, methodology): “Additional baselines include sigmoid classifiers deriving features from a CNN with 2D convolution and pooling operations [23], and RNNs (a GRU [45], LSTM and a Bi-LSTM) with a single hidden layer, max pooling and dropout.”
Neural Semantic Encoders (2016, methodology): “Model Yelp 13 IMDB Accuracy MSE Accuracy MSE Classifier with handcrafted features [31] 59.”
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers (2016, methodology): “These are orthogonal to each other, and it is possible to plug in the proposed ConvRec as a sentence feature extraction module in the Conv-GRNN from (Tang et al., 2015).”
A Co-Matching Model for Multi-choice Reading Comprehension (2018, methodology): “Then we build a bi-directional LSTM followed by max pooling on top of the co-matching states of each sentence as follows: 4) where the function MaxPooling ( · ) is the row-wise max pooling operation. is the sentence-level aggregation of the co-matching states.”
Towards Controllable Story Generation (2018, methodology): “We employ an LSTM-based logistic regression classiﬁer as it learns feature representations that capture long-term dependencies between the words, and has been shown efﬁcient in text classi-ﬁcation tasks (Tang et al., 2015).”
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings (2016, methodology): “, (Zhu et al., 2015; Tang et al., 2015; Tai et al., 2015; Le & Zuidema, 2015).”