Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification

By Duyu Tang, Furu Wei, Nan Yang, M. Zhou, Ting-Wei Liu, Bing Qin. 2014

1,223 citations recorded by the source corpus.

We present a method that learns word embedding for Twitter sentiment classification in this paper. Most existing algorithms for learning continuous word representations typically only model the syntactic context of words but ignore the sentiment of text. This is problematic for sentiment analysis as they usually map words with similar syntactic context but opposite sentiment polarity, such as good and bad, to neighboring word vectors. We address this issue by learning sentimentspecific word embedding (SSWE), which encodes sentiment information in the continuous representation of words. Specifically, we develop three neural networks to effectively incorporate the supervision from sentiment polarity of text (e.g. sentences or tweets) in their loss functions. To obtain large scale training corpora, we learn the sentiment-specific word embedding from massive distant-supervised tweets collected by positive and negative emoticons. Experiments on applying SSWE to a benchmark Twitter sentiment classification dataset in SemEval 2013 show that (1) the SSWE feature performs comparably with hand-crafted features in the top-performed system; (2) the performance is further improved by concatenating SSWE with existing feature set.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “[26] addresses this problem by proposing sentiment specific word embedding (SSWE).”
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification (2015, methodology): “We try to make better use of the sentiment information to learn a better SSWE (Tang et al., 2014), e.g. setting a large window size.”
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks (2014, methodology): “…on token-level applications (e.g., POS tagging) by Collobert et al. (2011), CNN has been used in systems for entity search, sentence modeling, word embedding learning, product feature mining, and so on (Xu and Sarikaya, 2013; Gao et al., 2014; Shen et al., 2014; Kalchbrenner et a r X i v : 1412 .”
Demographic Factors Improve Classification Performance (2015, methodology): “We follow Tang et al. (2014) in using convolutional layers over word embeddings (Collobert et al., 2011) to generate fixed-length input representations.”
Neural Sentiment Classification with User and Product Attention (2016, neutral): “SSWE generates features with sentiment-specific word embeddings (SSWE) (Tang et al., 2014) and then trains a SVM classifier.”
Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding (2015, methodology): “On text, CNN has been gaining attention, used in systems for tagging, entity search, sentence modeling, and so on [4, 5, 26, 7, 21, 12, 25, 22, 24, 13], to make use of the 1D structure (word order) of text data.”
Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation (2015, background): “…[Cho et al., 2014], syntactic parsing [Socher et al., 2013a], discourse parsing [Li et al., 2014], relation classification [Zeng et al., 2014], sentiment analysis [Socher et al., 2013c; Tang et al., 2014; Li, 2014], part-of-speech tagging and named entity recognition [Collobert et al., 2011].”
Hierarchical Attention Networks for Document Classification (2016, methodology): “SSWE uses sentiment specific word embeddings according to (Tang et al., 2014).”
Arming the public with artificial intelligence to counter social bots (2019, background): “Deep-learning approaches can also be used to detect sentiment online (Socher et al., 2013; Tang et al., 2014; Severyn and Moschitti, 2015), allowing the next generation of social bots to be more perceptive about human emotions.”
Effective LSTMs for Target-Dependent Sentiment Classification (2015, methodology): “SSWE h , SSWE r and SSWE u are different embedding learning algorithms introduced in (Tang et al., 2014).”
Gated Neural Networks for Targeted Sentiment Analysis (2016, background): “There has been other previous work on targeted sentiment analysis (Jiang et al. 2011; Tang et al. 2014; Dong et al. 2014), which we did not include in Table 4.”
Deep learning for sentiment analysis: A survey (2018, background): “Tang et al. 40 applied user representations and product representations in review classification.”
Distantly Supervised Lifelong Learning for Large-Scale Social Media Sentiment Analysis (2017, methodology): “[20] learned the sentiment-specific word embedding (SSWE) from massive distantly supervised tweets by developing three neural networks.”
emoji2vec: Learning Emoji Representations from their Description (2016, background): “Many current NLP systems applied to social media rely on representation learning and word embeddings (Tang et al., 2014; Dong et al., 2014; Dhingra et al., 2016; Augenstein et al., 2See https://twitter.com/Kyle_MacLachlan/ status/765390472604971009 for an extreme example. ar X iv :1 60 9.”
DeepHate: Hate Speech Detection via Multi-Faceted Text Representations (2020, methodology): “With the generated sentiment labels for the hate speech datasets, we train a sentiment-specific word embedding [37] by performing a sentiment classification task to predict the sentiment labels in hate speech datasets.”
Probabilistic variable-length segmentation of protein sequences for discriminative motif discovery (DiMotif) and sequence embedding (ProtVecX) (2018, background): “In particular, skip-gram neural networks combined with negative sampling 36 has resulted in the state-of-the-art performance in a broad range of NLP tasks 35 .”