Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification

By Jin Wang, Zhongyuan Wang, Dawei Zhang, Jun Yan. 2017

321 citations recorded by the source corpus.

Text classification is a fundamental task in NLP applications. Most existing work relied on either explicit or implicit text representation to address this problem. While these techniques work well for sentences, they can not easily be applied to short text because of its shortness and sparsity. In this paper, we propose a framework based on convolutional neural networks that combines explicit and implicit representations of short text for classification. We first conceptualize a short text as a set of relevant concepts using a large taxonomy knowledge base. We then obtain the embedding of short text by coalescing the words and relevant concepts on top of pre-trained word vectors. We further incorporate character level features into our model to capture fine-grained subword information. Experimental results on five commonly used datasets show that our proposed method significantly out-performs state-of-the-art methods.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xing Xie
2. Philip S. Yu
3. Hongwei Wang
4. Wenjie Li
5. Fuzheng Zhang
6. Miao Zhao
7. Erik Cambria
8. Minyi Guo

Citations in the corpus

• RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems (2018, methodology): “These knowledge graphs are successfully applied in many applications such as KG completion [14], question answering [7], word embedding [40], and text classification [34].”
• DKN: Deep Knowledge-Aware Network for News Recommendation (2018, methodology): “Distinct from existing work [46], KCNN is: 1) multi-channel, as it treats word embedding, entity embedding, and contextual entity embedding of news as multiple stacked channels just like colored images; 2) word-entity-aligned, as it aligns a word and its associated entity in multiple channels and ap”
• A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, background): “[165] augmented short text representation learning with knowledge-based conceptualization”
• A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “The SICK contains almost 10,000 English sentence pairs.”
• Deep Learning--based Text Classification (2020, methodology): “esent a tree-based CNN to capture sentencelevel semantics. Pang et al. [62] cast text matching as the image recognition task, and use multi-layer CNNs to identify salient n-gram patterns. Wang et al. [63] propose a CNN-based model that combines explicit and implicit , Vol. 1, No. 1, Article . Public”
• Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification (2019, methodology): “Previous studies have exploited latent topics (Zeng et al., 2018) and external knowledge (e.g., entities) from Knowledge Bases to enrich the semantics of the short texts (Wang et al., 2013, 2017).”
• A Survey on Text Classification: From Shallow to Deep Learning (2020, background): “P, etc. knowledge base [121] Accuracy TREC, Twitter, AG, Bing, MR 8âĂŘbit character encoding [122] Accuracy Geonames toponyms, etc. dynamic routing [78] Accuracy Subj, TREC, Reuters, etc. hierarchical relations [123] M icro − F 1, M acro − F 1, etc. RCV1, Amazon670K meta-learning [124] Accuracy…”