Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus

By Peter D. Turney, Michael L. Littman. arXiv.org, 2002

423 citations recorded by the source corpus.

The evaluative character of a word is called its semantic orientation. A positive semantic orientation implies desirability (e.g., "honest", "intrepid") and a negative semantic orientation implies undesirability (e.g., "disturbing", "superfluous"). This paper introduces a simple algorithm for unsupervised learning of semantic orientation from extremely large corpora. The method involves issuing queries to a Web search engine and using pointwise mutual information to analyse the results. The algorithm is empirically evaluated using a training corpus of approximately one hundred billion words — the subset of the Web that is indexed by the chosen search engine. Tested with 3,596 words (1,614 positive and 1,982 negative), the algorithm attains an accuracy of 80%. The 3,596 test words include adjectives, adverbs, nouns, and verbs. The accuracy is comparable with the results achieved by Hatzivassiloglou and McKeown (1997), using a complex four-stage supervised learning algorithm that is restricted to determining the semantic orientation of adjectives.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/5cb1e12c970db1cece45dc286e6de11b746d1c8a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lillian Lee
2. Bo Pang
3. Bing Liu
4. Songbo Tan
5. B. Liu
6. Xueqi Cheng
7. Huifeng Tang
8. Mike Thelwall

Citations in the corpus

• Building Lexicon for Sentiment Analysis from Massive Collection of HTML Documents (2007, methodology): “As a baseline, we built lexicon similarly by using polarity value of (Turney, 2002).”
• Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis (2004, background): “Much research is also being directed at acquiring affect lexica automatically (Turney 2002, Turney and Littman 2002).”
• Joint sentiment/topic model for sentiment analysis (2009, methodology): “Turney and Littman [22] applied an un-supervised learning algorithm to classify the semantic orientation in the word/phrase level, based on mutual information between document phrases and a small set of positive/negative paradigm words like “good” and “bad”.”
• Automatic Identification of Sentiment Vocabulary: Exploiting Low Association with Known Sentiment Terms (2005, background): “Movie and product reviews have been the main focus of many of the recent studies in this area (Pang and Lee 2002, Pang et al. 2004, Turney 2002, Turney and Littman 2002).”
• A survey on sentiment detection of reviews (2009, methodology): “Some studies showed that restricting features to those adjectives for word sentiment classification would improve performance (Andreevskaia & Bergler, 2006; Turney & Littman, 2002; Wiebe, 2000).”
• Contextual Valence Shifters (2006, background): “(Edmonds & Hirst 2002; Hatzivassiloglou and McKeown 1997; Turney& Littman 2002; Wiebe et al. 2001, In Press).”
• Mining the peanut gallery: opinion extraction and semantic classification of product reviews (2003, background): “Turney and Littman [23] determined the similarity between two words by counting the number of results returned by web searches joining the words with a NEAR operator.”
• A survey of multimodal sentiment analysis (2017, background): “Mutual Information in coocurences [63] and search results [64] with reference terms.”
• Pulse: Mining Customer Opinions from Free Text (2005, methodology): “We have begun experimenting with a variation of a technique for bootstrapping from seed words with known orientation [1,2] with promising initial results [15].”
• Thumbs up? Sentiment Classification using Machine Learning Techniques (2002, background): “We also note that Turney (2002) found movie reviews to be the most 2Indeed, although our choice of title was completely independent of his, our selections were eerily similar. difficult of several domains for sentiment classification, reporting an accuracy of 65.83% on a 120- document set…”
• Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses (2006, methodology): “5 % accuracy on the collapsed runs is comparable to the accuracy re-ported in the literature for other systems run on large corpora (Turney and Littman, 2002; Hatzi-vassiloglou and McKeown, 1997).”
• SENTIMENT CLASSIFICATION of MOVIE REVIEWS USING CONTEXTUAL VALENCE SHIFTERS (2006, methodology): “Research has also been done on positive/negative term-counting methods and automatically determining if a term is positive or negative (Turney and Littman 2002).”
• Senti-lexicon and improved Naïve Bayes algorithms for sentiment analysis of restaurant reviews (2012, background): “Turney and Littman (2002) and Kim and Hovy (2004) tried to find a sentiment word through the synonyms and antonyms by giving a seed word that clearly represented the sentiment, such as ‘‘good’’ or ‘‘bad’’.”