Text Graph Transformer for Document Classification

By Haopeng Zhang, Jiawei Zhang. 2020

75 citations recorded by the source corpus.

Text classification is a fundamental problem in natural language processing. Recent studies applied graph neural network (GNN) techniques to capture global word co-occurrence in a corpus. However, previous works are not scalable to large-sized corpus and ignore the heterogeneity of the text graph. To address these problems, we introduce a novel Transformer based heterogeneous graph neural network, namely Text Graph Transformer (TG-Transformer). Our model learns effective node representations by capturing structure and heterogeneity from the text graph. We propose a mini-batch text graph sampling method that significantly reduces computing and memory costs to handle large-sized corpus. Extensive experiments have been conducted on several benchmark datasets, and the results demonstrate that TG-Transformer outperforms state-of-the-art approaches on text classification task.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fei Wu
2. Jiwei Li
3. Qinghong Han
4. Yuxiao Lin
5. Xiaofei Sun
6. Kun Kuang
7. Yuxian Meng
8. Kyunghyun Cho

Citations in the corpus

• BertGCN: Transductive Text Classification by Combining GNN and BERT (2021, methodology): “Our work is inspired by the work of using graph neural networks for text classification (Yao et al., 2019; Huang et al., 2019; Zhang and Zhang, 2020).”
• A Survey on Text Classification: From Traditional to Deep Learning (2020, unknown): “2019 RoBERTa [140] arXiv SA, QA [141] F1, Accuracy SQuAD, MNLI-m, SST-2 Trans 2020 GAN-BERT [142] ACL SA, NLI [143] F1, Accuracy SST-5, MNLI 2020 BAE [144] EMNLP SA, QA [145] Accuracy Amazon, Yelp, MR, MPQA 2020 ALBERT [146] ICLR SA, QA [147] F1, Accuracy SST, MNLI, SQuAD 2020 TG-Transformer [148] E”