Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory

By Xin Wang, Yuanchao Liu, Chengjie Sun, Baoxun Wang, Xiaolong Wang. 2015

273 citations recorded by the source corpus.

In this paper, we introduce Long ShortTerm Memory (LSTM) recurrent network for twitter sentiment prediction. With the help of gates and constant error carousels in the memory block structure, the model could handle interactions between words through a flexible compositional function. Experiments on a public noisy labelled data show that our model outperforms several feature-engineering approaches, with the result comparable to the current best data-driven technique. According to the evaluation on a generated negation phrase test set, the proposed architecture doubles the performance of non-neural model based on bag-of-word features. Furthermore, words with special functions (such as negation and transition) are distinguished and the dissimilarities of words with opposite sentiment are magnified. An interesting case study on negation expression processing shows a promising potential of the architecture dealing with complex sentiment phrases.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “[24] proposed encoding entire”
Quasi-Recurrent Neural Networks (2016, background): “RNN applications in the natural language domain range from sentence classiﬁcation (Wang et al., 2015) to word-and character-level language modeling (Zaremba et al., 2014).”
Recurrent Dropout without Memory Loss (2016, background): “…to model and learn from sequential data, and for showing state-of-the-art results on various public benchmarks ranging from sentence classiﬁcation (Wang et al., 2015; Irsoy and Cardie, 2014; Liu et al., 2015) and various tagging problems (Dyer et al., 2015) to language modelling (Kim et al.,…”
A Primer on Neural Network Models for Natural Language Processing (2015, background): “For further discussion on the representation power of feed-forward neural networks, see (Bengio et al., 2015, Section 6.5).”
Embedding Logic Rules Into Recurrent Neural Networks (2019, methodology): “[15] used LSTM to generate a sentence representation for predicting the polarities of tweets.”
Natural Language Processing Advancements By Deep Learning: A Survey (2020, methodology): “Long Short-Term Memory (LSTM) recurrent models have also been utilized for tweet sentiment prediction [123].”
Deep learning for sentiment analysis: A survey (2018, methodology): “55,62,87,89,95”
Large-Scale Cross-Category Analysis of Consumer Review Content on Sales Conversion Leveraging Deep Learning (2019, unknown): “The first deep learning algorithm we implement is long shortterm memory recurrent neural networks (Wang et al. 2015), which works by taking word or character sequences as inputs.”
Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning (2016, background): “…chunking (Turian et al., 2010), part-of-speech tagging (Ling et al., 2015), dependency parsing (Lazaridou et al., 2013; Bansal et al., 2014; Dyer et al., 2015; Watanabe and Sumita, 2015), named entity recognition (Guo et al., 2014), and sentiment analysis (Socher et al., 2013; Wang et al., 2015).”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “Another limitation comes from learning embeddings based only on a small window of surrounding words, sometimes words such as good and bad share almost the same embedding [23], which is problematic if used in tasks such as sentiment analysis [24].”
Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model (2016, background): “…(NN) such as convolutional neural networks (CNN) (Kim, 2014; Kalchbrenner et al., 2014), recurrent neural networks (RNN) (Graves, 2012; Irsoy and Cardie, 2014) and long short-term memory (LSTM) (Wang et al., 2015; Liu et al., 2015) have been successfully employed for categorical sentiment analysis.”