A C-LSTM Neural Network for Text Classification

By Chunting Zhou, Chonglin Sun, Zhiyuan Liu, Francis Chi-Moon Lau. 2015

931 citations recorded by the source corpus.

Neural network models have been demonstrated to be capable of achieving remarkable performance in sentence and document modeling. Convolutional neural network (CNN) and recurrent neural network (RNN) are two mainstream architectures for such modeling tasks, which adopt totally different ways of understanding natural languages. In this work, we combine the strengths of both architectures and propose a novel and unified model called C-LSTM for sentence representation and text classification. C-LSTM utilizes CNN to extract a sequence of higher-level phrase representations, and are fed into a long short-term memory recurrent neural network (LSTM) to obtain the sentence representation. C-LSTM is able to capture both local features of phrases as well as global and temporal sentence semantics. We evaluate the proposed architecture on sentiment classification and question classification tasks. The experimental results show that the C-LSTM outperforms both CNN and LSTM and can achieve excellent performance on these tasks.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

Quasi-Recurrent Neural Networks (2016, methodology): “Zhou et al. (2015) apply CNNs at the word level to generate n -gram features used by an LSTM for text classiﬁcation.”
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “C-LSTM : A C-LSTM Neural Network for Text Classiﬁcation (Zhou et al., 2015).”
Bidirectional LSTM with attention mechanism and convolutional layer for text classification (2019, methodology): “C-LSTM: A model combining with the strengths of CNN and RNN for sentence representation and text classiﬁcation proposed by Zhou et al. [72] .”
DeepHate: Hate Speech Detection via Multi-Faceted Text Representations (2020, methodology): “[49] introduced the C-LSTM model, which stacked a convolutional neural network (CNN) and long-short term memory (LSTM) [22], for text classification.”
Deep Learning--based Text Classification (2020, neutral): “[129] propose a Convolutional LSTM (C-LSTM) network.”
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (2018, neutral): “For language only base-line models: ∪ CNN-LSTM (Zhou et al., 2015), RNTN (Socher et al., 2013 (Zilly et al., 2016).”
Target-Dependent Sentiment Classification With BERT (2019, methodology): “CNN has been successfully applied in sentence-level sentiment classification [16], [17], [50].”
BertGCN: Transductive Text Classification by Combining GNN and BERT (2021, background): “Different neural model architectures (Kim, 2014; Zhou et al., 2015; Radford et al., 2018; Chai et al., 2020) have demonstrated their effectiveness against traditional statistical feature based methods (Wallach, 2006).”
Predicting residential energy consumption using CNN-LSTM neural networks (2019, methodology): “The CNN-LSTM basically consists of Convolutional layer, Pooling layer, LSTM layer and Dense layer [43].”
Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models (2018, methodology): “Our best performing model CNN-LSTM is closely related to C-LSTM by Zhou et al. (2015), where an LSTM is laid on top of a CNN model.”
Text feature extraction based on deep learning: a review (2017, background): “In reference [104], CNN convolves and abstracts word vectors of the original text with filters of a certain length, and thus previous pure word vector become convolved abstract sequences.”
Natural Language Processing Advancements By Deep Learning: A Survey (2020, background): “Another unified architecture is the C-LSTM proposed in [102] for”