Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

By Linmei Hu, Tianchi Yang, Chuan Shi, Houye Ji, Xiaoli Li. Conference on Empirical Methods in Natural Language Processing, 2019

363 citations recorded by the source corpus.

Short text classification has found rich and critical applications in news and tweet tagging to help users find relevant information. Due to lack of labeled training data in many practical use cases, there is a pressing need for studying semi-supervised short text classification. Most existing studies focus on long texts and achieve unsatisfactory performance on short texts due to the sparsity and limited labeled data. In this paper, we propose a novel heterogeneous graph neural network based method for semi-supervised short text classification, leveraging full advantage of few labeled data and large unlabeled data through information propagation along the graph. In particular, we first present a flexible HIN (heterogeneous information network) framework for modeling the short texts, which can integrate any type of additional information as well as capture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph ATtention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. The attention mechanism can learn the importance of different neighboring nodes as well as the importance of different node (information) types to a current node. Extensive experimental results have demonstrated that our proposed model outperforms state-of-the-art methods across six benchmark datasets significantly.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/1ceba9b7499ef45eb3146c6cecd3b27e3bcacb13

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Xipeng Qiu
3. Pengfei Liu
4. Chuan Shi
5. Xuanjing Huang
6. Xiao Wang
7. Lichao Sun
8. Danqing Wang

Citations in the corpus

• A Survey on Text Classification: From Shallow to Deep Learning (2020, methodology): “ed self-attention layers by attending over its neighbors. Thus, some GAT-based models are proposed to compute the hidden representations of each node. The heterogeneous graph attention network (HGAT) [165] with a dual-level attention ACM Comput. Surv., Vol. 37, No. 4, Article 35. Publication date: J”
• A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “The Heterogeneous Graph ATtention networks (HGAT) [212] with a dual-level attention mechanism learns the importance of different neighboring nodes and node types in the current node.”
• A Survey on Deep Semi-Supervised Learning (2021, background): “Text classification AG News [95], BPpedia [96], Yahoo! Answers [97], IMDB [98], Yelp review [95], Snippets [99], Ohsumed [100], TagMyNews [101], MR [102], Elec [103], Rotten Tomatoes [102], RCV1 [104] [105], [106], [107], [108], [109], [110]”
• Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering (2020, background): “[ Wang et al. , 2019b; Hu et al. , 2019a] propose heterogeneous graph attention networks with dual-level attention mechanism.”
• Graph Transformer Networks (2019, background): “Also, as several heterogeneous graph datasets have been recently studied for other network analysis tasks, such as link prediction [36, 41] and graph classification [17, 24], applying our GTNs to the other tasks can be interesting future directions.”
• Heterogeneous Graph Structure Learning for Graph Neural Networks (2021, background): “…of multiple types of nodes and links (Shi et al. 2017), Heterogeneous Graph Neural Networks (HGNNs) are brought forward and have achieved remarkable improvements on series of applications (Wang et al. 2020a; Hu et al. 2019b; Li et al. 2019; Hu et al. 2019a; Fan et al. 2019a; Wang et al. 2020b).”
• Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020, background): “Linmei et al. (2019) focused on semi-supervised short text classification and constructed a topic-entity heterogeneous neural graph.”