Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks

By Ji Young Lee, Franck Dernoncourt. 2016

469 citations recorded by the source corpus.

Recent approaches based on artificial neural networks (ANNs) have shown promising results for short-text classification. However, many short texts occur in sequences (e.g., sentences in a document or utterances in a dialog), and most existing ANN-based systems do not leverage the preceding short texts when classifying a subsequent one. In this work, we present a model based on recurrent neural networks and convolutional neural networks that incorporates the preceding short texts. Our model achieves state-of-the-art results on three different datasets for dialog act prediction.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Neural Approaches to Conversational AI (2018, positive): “For short sentences where information has to be inferred from the context, Lee and Dernoncourt (2016) proposed to use recurrent and convolutional neural networks that also consider texts prior to the current utterance.”
Identifying Well-formed Natural Language Questions (2018, methodology): “Also, we used a single-layer word-level biLSTM encoder with hidden layer of length 50 to encode the question and then use this representation in the softmax layer to predict the label (Lee and Dernoncourt, 2016).”
NL2Type: Inferring JavaScript Function Types from Natural Language Information (2019, methodology): “They have been successfully applied to a number of problems in natural language processing that are similar to our classiﬁcation problem, such as sentiment analysis, which classiﬁes texts into different categories [13], [14], [19].”
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (2019, unknown): “, 2018) and modelling innovation (Artzi and Zettlemoyer, 2011; Henderson et al., 2013; Lee and Dernoncourt, 2016; hao Su et al., 2016; Dhingra et al., 2016; Li et al., 2016; Mrksic et al., 2017).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Hashemi et al. (2016) used a CNN to extract hierarchical text features for intent detection and illustrated the sequence classification capabilities of CNNs. Lee and Dernoncourt (2016) proposed a model for intent classification of short utterances.”
Black-Box Attacks against RNN based Malware Detection Algorithms (2017, methodology): “In future works we will use the proposed model to attack convolutional neural network (CNN) based malware detection algorithms, since many researchers have begun to use CNN to process sequential data recently [33, 16].”
De-identification of patient notes with recurrent neural networks (2016, methodology): “…language processing, such as language modeling (Mikolov et al., 2010), text classiﬁcation (Socher et al., 2013; Kim, 2014; Blunsom et al., 2014; Lee and Dernoncourt, 2016), question answering (We-ston et al., 2015; Wang and Nyberg, 2015), machine translation (Bahdanau et al., 2014; Tamura et…”
Restructuring Conversations using Discourse Relations for Zero-shot Abstractive Dialogue Summarization. (2019, background): “(CRF) (Okazaki, 2007) in this paper, which has been recognized as a good baseline for discourse labelling (Kumar et al., 2017; Ji et al., 2016; Lee and Dernoncourt, 2016).”
Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts (2018, unknown): “rent neural network (RNN) [33, 34, 35, 36].”
A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “60 - - - SeqTextRCNN [7] - - - - - - - - - oh-2LSTMp [75] - - 94.”
A Survey on Text Classification: From Shallow to Deep Learning (2020, background): “…classification – the procedure of designating pre-defined labels for text – is an essential and significant task in many Natural Language Processing (NLP) applications, such as sentiment analysis [1][2] [3], topic labeling [4] [5] [6], question answering [7] [8] and dialog act classification [9].”
HDLTex: Hierarchical Deep Learning for Text Classification (2017, methodology): “More recent work has used CNNs for text mining [17].”
Early Detection of Fake News on Social Media Through Propagation Path Classification with Recurrent and Convolutional Networks (2018, background): “A recent study (Lee and Dernoncourt 2016) adopts recurrent and convolutional networks for sequential short-text classification.”
Aspect-based Financial Sentiment Analysis with Deep Neural Networks (2018, background): “3191825 Recently, deep learning approaches have achieved a lot of success in many kinds of research including the sentence representation [9], text parsing [11], machine translation [2], and sentiment classification [5] [7].”
Long Document Classification From Local Word Glimpses via Recurrent Attention Learning (2019, background): “[11] J. Y. Lee and F. Dernoncourt.”