Graph Convolutional Networks for Text Classification

By Liang Yao, Chengsheng Mao, Yuan Luo. AAAI Conference on Artificial Intelligence, 2018

2,101 citations recorded by the source corpus.

Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

Subfields: Artificial Intelligence, Natural Language Processing, Graph Learning

View Paper: https://arxiv.org/abs/1809.05679

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Graph Learning

Citations in the corpus

Simplifying Graph Convolutional Networks (2019, methodology): “Yao et al. (2019) use one-hot features for the word and document nodes.”
Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology (2020, methodology): “We experiment with four groups of deep neural network based models for malevolent response detection and classification, namely CNN-based [28, 63], RNN-based [31, 34], GCN-based [59], and BERT-based [14].”
Hierarchical Multi-modal Contextual Attention Network for Fake News Detection (2021, methodology): “• TextGCN [39]: TextGCN employs the graph convolutional network to better learn words and document embeddings, in which the whole corpus is modeled as a heterogeneous graph.”
Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks (2019, methodology): “GCNs have been deployed on document-word relationships (Yao et al., 2018) and tree structures (Marcheggiani and Titov, 2017; Zhang et al.”
Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification (2020, methodology): “Actually, using GCN (Kipf and Welling, 2017) to encode the information conveyed by a dependency tree has already been investigated in several fields, e.g., modeling document-word relationships (Yao et al., 2019) and tree structures (Marcheggiani and Titov, 2017; Zhang et al., 2018).”
Text Graph Transformer for Document Classification (2020, methodology): “In addition, we compare with three state-of-the-art GNN-based models: TextGCN in (Yao et al., 2019) using GCN, Text GNN 3 in (Huang et al., 2019) using text level graphs and TensorGCN in (Liu et al., 2020) using semantic and syntactic contextual information.”
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks (2020, background): “Many GNN variants are proposed and also applied in various NLP tasks, such as text classification (Yao, Mao, and Luo 2019), machine translation (Marcheggiani, Bastings, and Titov 2018), dialogue policy optimization (Chen et al. 2018b; 2019; 2018a) etc.”
BertGCN: Transductive Text Classification by Combining GNN and BERT (2021, methodology): “Our work is inspired by the work of using graph neural networks for text classiﬁcation (Yao et al., 2019; Huang et al., 2019; Zhang and Zhang, 2020).”
Deep Learning--based Text Classification (2020, methodology): “Peng et al. [101]. for word and document, and then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Fig. 14. The architecture of GCNN [103]. Building GNNs for a large-scale text corpus is costly. There have been works on reducing the ”
Graph convolutional networks: a comprehensive review (2019, methodology): “Besides, TextGCN [96] models a whole corpus to a heterogeneous graph and learn word embedding and document embedding simultaneously, followed by a softmax classifier for text classification.”
Simple Spectral Graph Convolution (2021, methodology): “Following (Kipf & Welling, 2016), we trained our method and Text GCN for a maximum of 200 epochs using the where p ( i, j ) = W ( i,j ) W , p ( i ) = W ( i ) W .”
Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks (2021, neutral): “Recently, graph network-based models have achieve promising performance in many NLP tasks, such as text classification [24, 25], relation extraction [26], Chinese NER [27,28].”
A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses (2021, background): “Yao et al. (2019) propose text-GCN.”