Semantic Clustering and Convolutional Neural Network for Short Text Categorization

By Peng Wang, Jiaming Xu, Bo Xu, Chenglin Liu, Heng Zhang, Fangyuan Wang, Hongwei Hao. 2015

184 citations recorded by the source corpus.

Short texts usually encounter data sparsity and ambiguity problems in representations for their lack of context. In this paper, we propose a novel method to model short texts based on semantic clustering and convolutional neural network. Particularly, we first discover semantic cliques in embedding spaces by a fast clustering algorithm. Then, multi-scale semantic units are detected under the supervision of semantic cliques, which introduce useful external knowledge for short texts. These meaningful semantic units are combined and fed into convolutional layer, followed by max-pooling operation. Experimental results on two open benchmarks validate the effectiveness of the proposed method.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification (2015, background): “Convolutional Neural Networks (CNNs) have recently been shown to achieve impressive results on the practically important task of sentence categorization (Kim, 2014; Kalchbrenner et al., 2014; Wang et al., 2015; Goldberg, 2015; Iyyer et al., 2015).”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “(Wang et al., 2015b) proposed the usage of CNN for modeling representations of short texts, which suffer from the lack of available context and, thus, require extra efforts to create meaningful representations.”
A Primer on Neural Network Models for Natural Language Processing (2015, background): “…word (this is inspired by (Ling et al., 2015b)), an RNN that reads in a sentence and, based on the final state decides if it conveys positive or negative sentiment (this is inspired by (Wang et al., 2015b)) or an RNN that reads in a sequence of words and decides whether it is a valid noun-phrase.”
Understanding Convolutional Neural Networks for Text Classification (2018, background): “…for computer vision, have been shown to achieve strong performance on text clas-siﬁcation tasks (Bai et al., 2018; Kalchbrenner et al., 2014; Wang et al., 2015; Zhang et al., 2015; Johnson and Zhang, 2015; Iyyer et al., 2015) as well as other traditional Natural Language Processing (NLP)…”
Tempura: Query Analysis with Structural Templates (2020, methodology): “To date, such algorithms measure similarities with TF-IDF [1] or embeddingbased [40] distances, which usually result in query groups with mixed semantic (“when was the colored TV invented” v.”
Message Passing Attention Networks for Document Understanding (2019, background): “The concept of message passing over graphs has been around for many years (Weisfeiler and Lehman 1968; Murphy, Weiss, and Jordan 1999), as well as that of graph neural networks (GNNs) (Gori, Monfardini, and Scarselli 2005; Scarselli et al. 2008).”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “[50] proposed the usage of CNN for modeling representations of short texts, which suffer from the lack of available context and, thus, require extra efforts to create meaningful representations.”
Learning to Spot and Refactor Inconsistent Method Names (2019, background): “able results [43], [44] by managing to capture the semantics of sentences for relevant similarity computation.”