Automatically Assessing Review Helpfulness

By Soo-Min Kim, Patrick Pantel, Timothy Chklovski, Marco Pennacchiotti. Conference on Empirical Methods in Natural Language Processing, 2006

575 citations recorded by the source corpus.

User-supplied reviews are widely and increasingly used to enhance e-commerce and other websites. Because reviews can be numerous and varying in quality, it is important to assess how helpful each review is. While review helpfulness is currently assessed manually, in this paper we consider the task of automatically assessing it. Experiments using SVM regression on a variety of features over Amazon.com product reviews show promising results, with rank correlations of up to 0.66. We found that the most useful features include the length of the review, its unigrams, and its product rating.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/4492611e06d61d127d8b51d70d1be654a109209d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. B. Liu
2. Bing Liu
3. Yejin Choi
4. Mohit Bansal
5. Minlie Huang
6. Yongfeng Zhang
7. Jianfeng Gao
8. Erik Cambria

In Information Systems

1. B. Liu
2. Yongfeng Zhang
3. Arjun Mukherjee
4. Bing Liu
5. Leman Akoglu
6. Claire Cardie
7. Lei Zhang
8. Xu Chen

Citations in the corpus

• Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics (2011, background): “In parallel with our work, researchers in the natural language processing field have examined the task of predicting review helpfulness [23], [24], [25], [26], [27], using reviews from Amazon.”
• Sentiment Analysis and Subjectivity (2010, methodology): “A related problem that has also been studied in the past few years is the determination of the usefulness, helpfulness or utility of a review [31, 49, 57, 110].”
• NewsEmbed: Modeling News through Pre-trained Document Representations (2021, background): “Classical approaches involve statistical modelling of words to represent the whole document, such as bag-of-words [31], term frequency-inverse document frequency [32], and topic modeling [4].”
• Analysis of the Perceived Value of Online Tourism Reviews: Influence of Readability and Reviewer Characteristics (2016, result): “As prior research suggest (Hao et al., 2009; Jeon et al., 2006; Kim et al., 2006; Liu et al., 2007; Weimer & Gurevych, 2007), review length has a positive effect on the perceived helpfulness of a review.”
• Explaining Recommendations (2007, unknown): “This is confirmed by a similar study which attempts to predict review helpfulness in the domain of digital cameras and mp3 players, using SVM regression [13].”
• Quality evaluation of product reviews using an information quality framework (2011, methodology): “[16] measure the quality of a review in terms of the helpfulness votes submitted by readers.”
• Mining customer requirements from online reviews: A product improvement perspective (2016, methodology): “Kim et al. (2006) used the SVM regression on a variety of features over Amazon.com product reviews to assess the helpfulness of a review and obtained promising results.”
• What Yelp Fake Review Filter Might Be Doing? (2013, background): “Studies on review quality (Liu et al., 2007), distortion (Wu et al., 2010), and helpfulness (Kim et al., 2006) have also been conducted.”
• Predicting the “helpfulness” of online consumer reviews (2017, background): “Various research studies have been done on helpfulness of reviews (Kim et al., 2006).”
• Low-Quality Product Review Detection in Opinion Summarization (2007, methodology): “The problem of evaluating helpfulness of reviews (Kim et al, 2006), also known as learning utility of reviews (Zhang and Varadarajan, 2006), is quite similar to our problem of assessing the quality of reviews.”
• The Effectiveness of Personalized Movie Explanations: An Experiment Using Commercial Meta-data (2008, result): “Similar correlations between item rating and review helpfulness were found in other domains such as digital cameras and mobile phones [9].”
• A review of natural language processing techniques for opinion mining systems (2017, background): “Usefulness measurement of reviews aims to rank the reviews according to the degree of usefulness, which is usually formulated as a regression problem with the features of review lengths, review rating scores, POS tags, sentiment words, tf-idf weighting scores and so on [153, 154].”
• Finding Deceptive Opinion Spam by Any Stretch of the Imagination (2011, methodology): “Lastly, automatic approaches to determining review quality have been studied—directly (Weimer et al., 2007), and in the contexts of helpfulness (Danescu-Niculescu-Mizil et al., 2009; Kim et al., 2006; O’Mahony and Smyth, 2009) and credibility (Weerkamp and De Rijke, 2008).”
• A Survey of Opinion Mining and Sentiment Analysis (2012, background): “In [48-50], researchers also built models to identify some specific types of opinions.”
• Opinion Mining and Sentiment Analysis (2008, methodology): “For example, 38% of a sample of roughly 20,000 Amazon MP3-player reviews, and 31% of those aged at least three months, received three or fewer utility evaluations [161].”