Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales

By Bo Pang, Lillian Lee. 2005

3,193 citations recorded by the source corpus.

We address the rating-inference problem, wherein rather than simply decide whether a review is "thumbs up" or "thumbs down", as in previous sentiment analysis work, one must determine an author's evaluation with respect to a multi-point scale (e.g., one to five "stars"). This task represents an interesting twist on standard multi-class text categorization because there are several different degrees of similarity between class labels; for example, "three stars" is intuitively closer to "four stars" than to "one star".We first evaluate human performance at the task. Then, we apply a meta-algorithm, based on a metric labeling formulation of the problem, that alters a given n-ary classifier's output in an explicit attempt to ensure that similar items receive similar labels. We show that the meta-algorithm can provide significant improvements over both multi-class and regression versions of SVMs when we employ a novel similarity measure appropriate to the problem.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Layer Normalization (2016, background): “…50,000 iterations and evaluate their performance on ﬁve tasks: semantic-relatedness (SICK) [Marelli et al., 2014], movie review sentiment (MR) [Pang and Lee, 2005], customer product reviews (CR) [Hu and Liu, 2004], subjectivity/objectivity classiﬁcation (SUBJ) [Pang and Lee, 2004] and opinion…”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, methodology): “Many of the SentEval tasks are closely re-lated to sentiment analysis, such as MR (Pang and Lee, 2005), SST (Socher et al., 2013), CR (Hu and Liu, 2004), and SUBJ (Pang and Lee, 2004).”
An efficient framework for learning sentence representations (2018, methodology): “The following classiﬁcation benchmarks are commonly used: movie review sentiment (MR) (Pang & Lee, 2005), product reviews (CR) (Hu & Liu, 2004), subjectivity classiﬁcation (SUBJ) (Pang & Lee, 2004), opinion polarity (MPQA) (Wiebe et al., 2005), question type classiﬁ-cation (TREC) (Voorhees &…”
Semi-supervised Sequence Learning (2015, methodology): “In our experiments on document classification with 20 Newsgr oups [16] and DBpedia [19], and sentiment analysis with IMDB [21] and Rotten Tomatoes [25], LSTMs pretrained by recurrent language models or sequence autoencoders are usually better th an LSTMs initialized randomly.”
Skip-Thought Vectors (2015, methodology): “We use 5 datasets: movie review sentiment (MR) [36], customer product reviews (CR) [37], sub-jectivity/objectivity classiﬁcation (SUBJ) [38], opinion polarity (MPQA) [39] and question-type classiﬁcation (TREC) [40].”
Distributed Representations of Sentences and Documents (2014, methodology): “Dataset: This dataset was first proposed by (Pang & Lee, 2005) and subsequently extended by (Socher et al., 2013b) as a benchmark for sentiment analysis.”
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013, methodology): “We consider the corpus of movie review excerpts from the rottentomatoes.com website originally collected and published by Pang and Lee (2005).”
SentEval: An Evaluation Toolkit for Universal Sentence Representations (2018, background): “…tasks (see Table 1) that covers various types of sentence classiﬁcation, including sentiment analysis (MR and both binary and ﬁne-grained SST) (Pang and Lee, 2005; Socher et al., 2013), question-type (TREC) (Voorhees and Tice, 2000), product reviews (CR) (Hu and Liu, 2004),…”
Adversarial Training Methods for Semi-Supervised Text Classification (2016, background): “For the Rotten Tomatoes dataset, for each step, we take a batch of size 64 for calculating the loss of the negative log-likelihood and adversarial training, and 512 for calculating the loss of virtual adversarial training.”
Representation Learning with Contrastive Predictive Coding (2018, methodology): “For the classiﬁcation tasks we used the following datasets: movie review sentiment (MR) [43], customer product reviews (CR) [44], subjectivity/objectivity [45], opinion polarity (MPQA) [46] and question-type classiﬁcation (TREC) [47].”
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, background): “(iv) TREC-50: a fine-grained version of the same TREC-6 question classification task with 50 classes (v) SUBJ: subjective/objective binary classification dataset (Pang and Lee, 2005).”
Convolutional Neural Networks for Sentence Classification (2014, neutral): “Classification involves detecting positive/negative reviews (Pang and Lee, 2005).”
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019, methodology): “We compare the SBERT sentence embeddings to other sentence embeddings methods on the following seven SentEval transfer tasks: • MR : Sentiment prediction for movie reviews snippets on a ﬁve start scale (Pang and Lee, 2005).”
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks (2015, methodology): “We use the raw Cornell ﬁlm reviews corpus collected by Pang and Lee [27].”
Learning to Generate Reviews and Discovering Sentiment (2017, methodology): “On the MR (Pang & Lee, 2005) and CR (Hu & Liu, 2004) sentiment analysis datasets we improve the state of the art by a signiﬁcant margin.”