Measuring praise and criticism: Inference of semantic orientation from association

By Peter D. Turney, Michael L. Littman. 2003

1,770 citations recorded by the source corpus.

The evaluative character of a word is called its semantic orientation. Positive semantic orientation indicates praise (e.g., "honest", "intrepid") and negative semantic orientation indicates criticism (e.g., "disturbing", "superfluous"). Semantic orientation varies in both direction (positive or negative) and degree (mild to strong). An automated system for measuring semantic orientation would have application in text classification, text filtering, tracking opinions in online discussions, analysis of survey responses, and automated chat systems (chatbots). This article introduces a method for inferring the semantic orientation of a word from its statistical association with a set of positive and negative paradigm words. Two instances of this approach are evaluated, based on two different statistical measures of word association: pointwise mutual information (PMI) and latent semantic analysis (LSA). The method is experimentally tested with 3,596 words (including adjectives, adverbs, nouns, and verbs) that have been manually labeled positive (1,614 words) and negative (1,982 words). The method attains an accuracy of 82.8% on the full test set, but the accuracy rises above 95% when the algorithm is allowed to abstain from classifying mild words.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Saif M. Mohammad
2. B. Liu
3. Erik Cambria
4. Rada Mihalcea
5. Yejin Choi
6. Mohit Bansal
7. Bing Liu
8. Maite Taboada

Citations in the corpus

• SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (2020, methodology): “Four models with different inductive biases were used: PMI (Turney and Littman, 2003), FastText (Joulin et al., 2016), LSTM (Hochreiter and Schmidhuber, 1997), and BERT (Devlin et al., 2019).”
• Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases (2020, background): “Prior work on lexicon induction methods compensates for the lack of existing annotated data on valence (Hatzivassiloglou and McKeown 1997; Riloff and Wiebe 2003; Turney and Littman 2003).”
• SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining (2006, methodology): “…or a Negative connotation (Esuli and Sebastiani, 2005; Hatzivassiloglou and McKeown, 1997; Kamps et al., 2004; Kim and Hovy, 2004; Takamura et al., 2005; Turney and Littman, 2003), since it is by considering the combined contribution of these terms that one may hope to solve Tasks 1, 2 and 3.”
• Creating Subjective and Objective Sentence Classifiers from Unannotated Texts (2005, methodology): “Automatic subjectivity/opinion/sentiment analysis is being applied to many interesting applications, including classification of reviews [19, 14, 11, 25, 26], analysis of product reputations [26, 25, 27], tracking sentiments toward events [28, 22, 29], and incorporating opinions into into question a”
• Mine Your Own Business: Market-Structure Surveillance Through Text Mining (2012, background): “Such normalization is called lift (or point-wise mutual information; see Turney and Littman 2003).”
• Corpus-based Learning of Analogies and Semantic Relations (2005, methodology): “As mentioned in Section 2.1, the VSM algorithm performs as well as an ensemble of twelve other modules (Turney et al., 2003).”
• Sentiment Analysis using Support Vector Machines with Diverse Information Sources (2004, background): “In the past, work has been done in the area of characterizing words and phrases according to their emotive tone (Turney and Littman, 2003; Turney, 2002; Kamps et al., 2002; Hatzivassiloglou and Wiebe, 2000; Hatzi-vassiloglou and McKeown, 2002; Wiebe, 2000), but in many domains of text, the values…”
• Learning and Evaluating Emotion Lexicons for 91 Languages (2020, background): “Early work tended to focus on word statistics, often in combination with linguistic rules (Hatzivassiloglou and McKeown, 1997; Turney and Littman, 2003).”
• Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon (2010, methodology): “We will also use it to evaluate automatically generated lexicons, such as the polarity lexicons by Turney and Littman (2003) and Mohammad et al. (2009).”
• Articles: Recognizing Contextual Polarity: An Exploration of Features for Phrase-Level Sentiment Analysis (2009, background): “…from work on learning the prior polarity (semantic orientation) of words and phrases (e.g., Hatzivassiloglou and McKeown 1997; Kamps and Marx 2002; Turney and Littman 2003; Hu and Liu 2004; Kim and Hovy 2004; Esuli and Sebastiani 2005; Takamura, Inui, and Okumura 2005; Popescu and Etzioni 2005;…”
• The evolution of sentiment analysis - A review of research topics, venues, and top cited papers (2016, background): “The International Conference on Recent Advances in Natural Language Processing (RANLP) is an international conference organized by the Association for Computational Linguistics (ACL).”