Recurrent Convolutional Neural Networks for Text Classification

By Siwei Lai, Liheng Xu, Kang Liu, Jun Zhao. 2015

2,545 citations recorded by the source corpus.

Text classification is a foundational task in many NLP applications. Traditional text classifiers often rely on many human-designed features, such as dictionaries, knowledge bases and special tree kernels. In contrast to traditional methods, we introduce a recurrent convolutional neural network for text classification without human-designed features. In our model, we apply a recurrent structure to capture contextual information as far as possible when learning word representations, which may introduce considerably less noise compared to traditional window-based neural networks. We also employ a max-pooling layer that automatically judges which words play key roles in text classification to capture the key components in texts. We conduct experiments on four commonly used datasets. The experimental results show that the proposed method outperforms the state-of-the-art methods on several datasets, particularly on document-level datasets.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient (2016, background): “Deep discriminative models such as deep neural network (DNN) (Vesel`y et al. 2013), convolutional neural network (CNN) (Kim 2014) and recurrent convolutional neural network (RCNN) (Lai et al. 2015) have shown a high performance in complicated sequence classiﬁcation tasks.”
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks (2016, methodology): “(4) This max-pooling technique has proven effective in text classification tasks (Lai et al., 2015).”
Hierarchical Attention Networks for Document Classification (2016, background): “There are also some works that combine LSTM and CNN structure to for sentence classiﬁcation (Lai et al., 2015; Zhou et al., 2015).”
Self-Adaptive Hierarchical Sentence Model (2015, background): “For bidirectional recurrent neural networks, the reader is referred to [Lai et al., 2015] for more details.”
DKN: Deep Knowledge-Aware Network for News Recommendation (2018, background): “7Researchers have also proposed other types of neural network models for sentence modeling such as recurrent neural networks [40], recursive neural networks [38], and hybrid models [24].”
A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification (2015, methodology): “To this end, we augmented this representation with averaged word vectors (from Google word2vec3 or GloVe4) calculated over the words comprising the sentence, similar to the method in (Lai et al., 2015).”
FRAGE: Frequency-Agnostic Word Representation (2018, methodology): “Following the setting in [22], we implement a Recurrent CNN-based model11 and test it on AG’s news corpus (AGs), IMDB movie review dataset (IMDB) and 20 Newsgroups (20NG).”
Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, methodology): “• TextRCNN (Lai et al., 2015) combines both RNN and CNN for text classification tasks.”
Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN (2018, methodology): “We compare with XML-CNN model [29] which considered multi-label text classification using simple CNN model originally applied to sentence classification [20] and Recurrent Convolutional Neural Network model (RCNN) [23].”
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “Compared with RCNN (Lai et al., 2015), BLSTM2DCNN achieves a comparable result.”
MathDQN: Solving Arithmetic Word Problems via Deep Reinforcement Learning (2018, background): “…(Socher et al. 2011), recursive neural tensor network (Socher et al. 2013b), convolutional neural networks (Kim 2014), DCNN with dynamic k-max pooling (Kalchbrenner et al. 2014), recurrent convolutional neural network (Lai et al. 2015) and even character-level deep models (Zhang et al. 2015).”
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings (2016, background): “Another related work is (Lai et al., 2015), which combined pooling with non-LSTM recurrent networks and a word embedding.”
Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology (2020, methodology): “Text-RNN uses the last hidden state [34], while text-recurrent convolutional neural network (RCNN) uses the combination of the hidden states by adding CNN based modules on RNN outputs to better capture sequential information [31].”
Towards Explainable NLP: A Generative Explanation Framework for Text Classification (2018, methodology): “A common way to do text classiﬁcation tasks is using an Encoder-Predictor architecture (Zhang et al., 2015; Lai et al., 2015).”
Topic-based Evaluation for Conversational Bots (2018, background): “Deep learning advances have had a profound influence on natural language and dialog research, and researchers have experimented with Recurrent Neural Networks (RNNs)[11], [20], [13] and Convolutional Neural Networks (CNNs) [9], [4], [8] for text classification and summarization tasks.”
Hierarchical CVAE for Fine-Grained Hate Speech Classification (2018, neutral): “Lai et al. (2015) and Zhou et al. (2015) further combine RNN with CNN. Tang et al. (2015) and Yang et al. (2016) exploit the hierarchical structure for document classification.”
Bidirectional LSTM with attention mechanism and convolutional layer for text classification (2019, background): “RNNs is a biased model, where later words are more dominant than earlier words [60] .”