Uncovering ChatGPT’s Capabilities in Recommender Systems
By Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, ZhongXiang Sun, Xiao Zhang, Jun Xu. ACM Conference on Recommender Systems, 2023
389 citations recorded by the source corpus.
The debut of ChatGPT has recently attracted significant attention from the natural language processing (NLP) community and beyond. Existing studies have demonstrated that ChatGPT shows significant improvement in a range of downstream NLP tasks, but the capabilities and limitations of ChatGPT in terms of recommendations remain unclear. In this study, we aim to enhance ChatGPT’s recommendation capabilities by aligning it with traditional information retrieval (IR) ranking capabilities, including point-wise, pair-wise, and list-wise ranking. To achieve this goal, we re-formulate the aforementioned three recommendation policies into prompt formats tailored specifically to the domain at hand. Through extensive experiments on four datasets from different domains, we analyze the distinctions among the three recommendation policies. Our findings indicate that ChatGPT achieves an optimal balance between cost and performance when equipped with list-wise ranking. This research sheds light on a promising direction for aligning ChatGPT with recommendation tasks. To facilitate further explorations in this area, the full code and detailed original results are open-sourced at https://github.com/rainym00d/LLM4RS.
Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/450b5490cc653478c272be50aa986798df828a20
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Information Systems
In Management Science and Operations Research
Citations in the corpus
- When large language models meet personalization: perspectives of challenges and opportunities (2023, neutral): “[144] text-davinci-002 text-davinci-003 gpt-3.”
- Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting (2023, background): “A concurrent work (Dai et al., 2023) studied pairwise prompting in recommender systems, which is a substantially different application and their method still largely fall behind state-of-the-art models with sufficient data.”
- Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, neutral): “• Vague intention ( 𝐼 1 ) .”
- LLMRec: Large Language Models with Graph Augmentation for Recommendation (2023, background): “Future directions for investigation include integrating causal inference into side information debiasing and exploring counterfactual factors for context-aware user preference.”
- A Survey on Knowledge Distillation of Large Language Models (2024, methodology): “…from LLMs can benefit each component of the IR system, including the Query Rewriter (Srinivasan et al., 2022; Ma et al., 2023c), the Retriever (Dai et al., 2023b; Sachan et al., 2022, 2023; Schick and Sch¨utze, 2021; Meng et al., 2023; Peng et al., 2023b), and the Reranker (Bonifacio et al.,…”
- Tool learning with large language models: a survey (2024, neutral): “In this section, we provide an overview of the key concepts and terminology related to tool learning, offering a clearer understanding of the fundamental aspects of this field.”
- Large Language Models are Zero-Shot Rankers for Recommender Systems (2023, background): “Indeed, there are some preliminary attempts that employ LLMs for solving recommendation tasks [9, 29, 30, 5, 21, 34].”
- Recommender Systems in the Era of Large Language Models (LLMs) (2023, neutral): “ChatGPT [4] In-context Learning (ICL) Top-K Recommendation ChatGPT [3], [38], [82], [83] [84], [85], [86], [”