Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN

By Hao Peng, Jianxin Li, Yu He, Yaopeng Liu, Mengjiao Bao, Lihong Wang, Yangqiu Song, Qiang Yang. 2018

436 citations recorded by the source corpus.

Text classification to a hierarchical taxonomy of topics is a common and practical problem. Traditional approaches simply use bag-of-words and have achieved good results. However, when there are a lot of labels with different topical granularities, bag-of-words representation may not be enough. Deep learning models have been proven to be effective to automatically learn different levels of representations for image data. It is interesting to study what is the best way to represent texts. In this paper, we propose a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-of-words representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels. Our results on both RCV1 and NYTimes datasets show that we can significantly improve large-scale hierarchical text classification over traditional hierarchical text classification and existing deep models.

Subfields: Artificial Intelligence, Natural Language Processing, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Graph Learning

Citations in the corpus

Graph Convolutional Networks for Text Classification (2018, background): “Some recent studies explored graph neural networks for text classiﬁcation (Henaff, Bruna, and LeCun 2015; Defferrard, Bresson, and Vandergheynst 2016; Kipf and Welling 2017; Peng et al. 2018; Zhang, Liu, and Song 2018).”
Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology (2020, neutral): “First, graph neural networks (GCNs) have drawn the attention of researchers, with various methods that build graphs and do graph feature engineering [32, 42].”
Message Passing Attention Networks for Document Understanding (2019, methodology): “(Peng et al. 2018) use a GNN for hierarchical classification into a large taxonomy of topics.”
Text Level Graph Neural Network for Text Classification (2019, methodology): “With the development of GNN, some graphbased classification models are gradually emerging (Hamilton et al., 2017; Veličković et al., 2017; Peng et al., 2018).”
A Survey on Text Classification: From Shallow to Deep Learning (2020, methodology): “…modifications on BERT [87] SQuAD, MNLI-m, SST-2 F1, Accuracy improvement of BERT [89] F1, Accuracy SST, MNLI, SQuAD graph-CNN for multi-label text [68] M icro − F 1, M acro − F 1, etc. RCV1, NYTimes build a heterogeneous graph [6] Accuracy 20NG, Ohsumed, R52, R8, MR GNN removing the…”
Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting (2020, methodology): “Graph Processing Similar to the standard process in learning convolutional neural networks for graphs [41, 42, 43], the converting from graph data to matrix contains four major parts, the incidence graph generation unit, the subgraph generation unit, normalized matrix unit and the traffic flows matr”
Graph convolutional networks: a comprehensive review (2019, methodology): “In [98], a graph-of-words is constructed to capture long-distance semantics, and then, a recursively regularized graph convolution model is applied to leverage the hierarchy of labels.”
A Survey on Text Classification: From Traditional to Deep Learning (2020, methodology): “DGCNN [153] is a graph-CNN converting text to graph-ofwords, having the advantage of learning different levels of semantics with CNN models.”
Text Graph Transformer for Document Classification (2020, background): “Recently, graph based text classiﬁcation models received growing attention due to its ability to model global information in corpus (Yao et al., 2019; Peng et al., 2018; Zhang et al., 2020b; Nikolentzos et al., 2019).”
Self-Attention Graph Pooling (2019, background): “…which include recommender systems (van den Berg et al., 2017; Yao & Li, 2018; Monti et al., 2017), chemical researches (You et al., 2018; Zitnik et al., 2018), natural language processing (Bastings et al., 2017; Peng et al., 2018; Yao et al., 2018), and in many tasks as reported in Zhou et al..”
A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses (2021, methodology): “First, graph neural networks (GNNs) have drawn the attention of researchers, with various methods that build graphs and do graph feature engineering (Levy & Goldberg, 2014; Peng et al., 2018).”
Deep Learning--based Text Classification (2020, methodology): “[105] propose a graph-CNN-based DL model to first convert text to graph-of-words, and then use graph convolution operations to convolve the word graph, as shown in Figure 10.”
Weakly-Supervised Hierarchical Text Classification (2018, methodology): “A graph-CNN based deep learning model is proposed in (Peng et al. 2018) to convert text to graph-of-words, on which the graph convolution operations are applied for feature extraction.”
Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Others [23, 97, 196, 254, 256] train a unified classifier for the entire taxonomy.”