Sentiment Analysis in the Era of Large Language Models: A Reality Check

By Wenxuan Zhang, Yue Deng, Bing-Quan Liu, Sinno Jialin Pan, Lidong Bing. NAACL-HLT, 2023

629 citations recorded by the source corpus.

Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, \textsc{SentiEval}, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at \url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/c589ddc6c6fb07189af7c1212f6eb15c5ff72cde

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jindong Wang
2. Xingxu Xie
3. Hao Chen
4. Yidong Wang
5. Linyi Yang
6. Weirong Ye
7. Yue Zhang
8. Xiaoyuan Yi

Citations in the corpus

• Large Language Models in Finance: A Survey (2023, background): “LLMs can perform tasks through zero-shot learning [26], as demonstrated by their satisfactory performance in sentiment classification tasks across complex levels [65].”
• Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers (2024, neutral): “Templates for Task Implementation For different models, we apply different templates shown in Table 8, 9 and 10, referring to the previous works (Iyer et al., 2022; Taori et al., 2023; Zhang et al., 2023b; Li et al., 2023; Fu et al., 2023).”
• A Survey on Evaluation of Large Language Models (2023, background): “In low-resource learning environments, LLMs exhibit significant advantages over small language models (Zhang et al., 2023d), but the ability of ChatGPT to”
• Small Encoders Can Rival Large Decoders in Detecting Groundedness (2025, unknown): “This tradeoff between model size and efficiency aligns with findings from [Zhang et al., 2023, Zim-merman et al., 2024], which emphasize the advantages of compact, well-trained encoders in classification tasks.”
• The Price of Prompting: Profiling Energy Use in Large Language Models Inference (2024, unknown): “In the current literature, several endeavors aim to quantify and mitigate the environmental impact of AI.”