VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text
By Clayton J. Hutto, Eric Gilbert. 2014
5,811 citations recorded by the source corpus.
The inherent nature of social media content poses serious challenges to practical applications of sentiment analysis. We present VADER, a simple rule-based model for general sentiment analysis, and compare its effectiveness to eleven typical state-of-practice benchmarks including LIWC, ANEW, the General Inquirer, SentiWordNet, and machine learning oriented techniques relying on Naive Bayes, Maximum Entropy, and Support Vector Machine (SVM) algorithms. Using a combination of qualitative and quantitative methods, we first construct and empirically validate a gold-standard list of lexical features (along with their associated sentiment intensity measures) which are specifically attuned to sentiment in microblog-like contexts. We then combine these lexical features with consideration for five general rules that embody grammatical and syntactical conventions for expressing and emphasizing sentiment intensity. Interestingly, using our parsimonious rule-based model to assess the sentiment of tweets, we find that VADER outperforms individual human raters (F1 Classification Accuracy = 0.96 and 0.84, respectively), and generalizes more favorably across contexts than any of our benchmarks.
Subfields: Artificial Intelligence, Information Systems, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
- Noah A. Smith
- Yoshua Bengio
- Jason Aaron Edward Weston
In Information Systems
- Jure Leskovec
- Srijan Kumar
- William L. Hamilton
- Leon Derczynski
- Xiang Ren
- Ralf Krestel
- Julian Risch
- Marcos Zampieri
In Natural Language Processing
- Jason Aaron Edward Weston
- Mohit Bansal
- Yejin Choi
- Luke S. Zettlemoyer
- Noah A. Smith
- Jianfeng Gao
- Percy Liang
- Adina Williams
Citations in the corpus
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, methodology): “VADER produces a sentiment score between -1 and 1.”
- The Woman Worked as a Babysitter: On Biases in Language Generation (2019, methodology): “Moreover, we calculate Spearman’s correlation between 1) sentiment annotations and regard an-6 notations, 2) VADER predictions and sentiment annotations, and 3) VADER predictions and regard annotations in Table 5.”
- Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “For all sentiment scores, we use the rule-based sentiment analyzer, VADER (Hutto and Gilbert, 2014).”
- Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, methodology): “We used the VADER sentiment lexicon (Hutto and Gilbert, 2014), the SocialSent lexicons (Hamilton et al., 2016), and a small manually-created one using the words from the UNQOVER questions above.”
- Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “We analyze and report metrics for three methods: • Sentiment analysis: Following Sheng et al. (2021), we use VADER (Hutto & Gilbert, 2014), a rule-based sentiment analyzer, utilizing the implementation provided by NLTK (Xue, 2011).”
- BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (2021, methodology): “Sentiment analysis is commonly used to analyze sentiments in a customer’s reviews or opinions in social media [13, 24].”
- Code Llama: Open Foundation Models for Code (2023, methodology): “The VADER produces sentiment scores between -1 and 1, where a positive (negative) score indicates a positive (negative) sentiment towards the population mentioned in the prompt.”
- Does Gender Matter? Towards Fairness in Dialogue Systems (2019, methodology): “We use the public sentiment analysis tool Vader (Hutto and Gilbert, 2014) to predict the sentiment of a given response.”
- Ethical Challenges in Data-Driven Dialogue Systems (2017, background): “Sentiment [19] (compound scale from negative sentiment=-1 to positive sentiment=1), Flesch-Kincaid readability [19] (higher”
- Understanding and Evaluating Racial Biases in Image Captioning (2021, methodology): “For the frst column the VADER sentiment score [36] is multiplied by 100.”
- Automated Hate Speech Detection and the Problem of Offensive Language (2017, methodology): “We also use a sentiment lexicon designed for social media to assign sentiment scores to each tweet (Hutto and Gilbert 2014).”
- Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, methodology): “For sentiment analysis, we use the VADER sentiment analysis tool (Gilbert and Hutto, 2014) which is a”
- Statistical Arbitrage in Cryptocurrency Markets (2019, background): “Further information on this algorithm can be found in Gilbert (2014).”
- Revealing Persona Biases in Dialogue Systems (2021, positive): “Here, we also mask demographic mentions in the response and use VADER to automatically label the sentiment of the response.”
- Dats Wassup!!: Investigating African-American Vernacular English in Transformer-Based Text Generation (2020, methodology): “In addition to the DistilBERT sentiment classifier, we use VADER, which is a lexicon and rule-based sentiment analysis tool that is attuned to social-media specific sentiment intensity (Hutto and Gilbert, 2015), and TextBlob3, which does not have documentation on its implementation.”
- SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (2020, methodology): “…n -grams character n -grams lexicons for sentiment analysis and lexicon of offensive words Other representations included emoji priors extracted from the weakly supervised dataset, and sentiment analysis using NLTK (Bird et al., 2009), Vader (Hutto and Gilbert, 2014) and FLAIR (Akbik et al., 2018).”