TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation

By Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, Xiangnan He. ACM Conference on Recommender Systems, 2023

782 citations recorded by the source corpus.

Large Language Models (LLMs) have demonstrated remarkable performance across diverse domains, thereby prompting researchers to explore their potential for use in recommendation systems. Initial attempts have leveraged the exceptional capabilities of LLMs, such as rich knowledge and strong generalization through In-context Learning, which involves phrasing the recommendation task as prompts. Nevertheless, the performance of LLMs in recommendation tasks remains suboptimal due to a substantial disparity between the training tasks for LLMs and recommendation tasks, as well as inadequate recommendation data during pre-training. To bridge the gap, we consider building a Large Recommendation Language Model by tunning LLMs with recommendation data. To this end, we propose an efficient and effective Tuning framework for Aligning LLMs with Recommendations, namely TALLRec. We have demonstrated that the proposed TALLRec framework can significantly enhance the recommendation capabilities of LLMs in the movie and book domains, even with a limited dataset of fewer than 100 samples. Additionally, the proposed framework is highly efficient and can be executed on a single RTX 3090 with LLaMA-7B. Furthermore, the fine-tuned LLM exhibits robust cross-domain generalization. Our code and data are available at https://github.com/SAI990323/TALLRec.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/3487c12512fa41d3a4d64f00cb842525a8590ad3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wayne Xin Zhao
2. Ji-rong Wen
3. Yupeng Hou
4. Hongyu Lu
5. Bowen Zheng
6. Yu Chen
7. Ruobing Xie
8. Junjie Zhang

In Information Systems

1. Wayne Xin Zhao
2. Yupeng Hou
3. Ji-rong Wen
4. Hongyu Lu
5. Bowen Zheng
6. Yu Chen
7. Ruobing Xie
8. Junjie Zhang

Citations in the corpus

• Representation Learning with Large Language Models for Recommendation (2023, background): “These limitations can be attributed to three key factors: i) The hallucination issue of LLMs, where they may suggest recommended items that are not included in the candidate set; ii) The lack of a comprehensive global text-based collaborative relationship input due to the token limit of LLMs; iii)…”
• Recommender Systems in the Era of Large Language Models (LLMs) (2023, positive): “In recommender systems, PEFT can greatly reduce the computational cost of fine-tuning LLMs for recommendation tasks, which requires less update and maintains most of the model capabilities.”
• WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions (2023, neutral): “(4) Open-source models trained from Llama 13B, including Baize (Xu et al., 2023), CAMEL (Li et al., 2023a), and Tulu (Wang et al., 2023)”
• LLMRec: Large Language Models with Graph Augmentation for Recommendation (2023, background): “Future directions for investigation include integrating causal inference into side information debiasing and exploring counterfactual factors for context-aware user preference.”
• When large language models meet personalization: perspectives of challenges and opportunities (2023, positive): “TALLRe [119] explores the ability of large language models for the sequential recommendation.”
• Large Language Models are Zero-Shot Rankers for Recommender Systems (2023, methodology): “Here we would like to investigate whether the ability to rank items according to historical interactions using LLMs can be improved by instruction tuning.”
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, positive): “For this purpose, we automatically generate a large number of recommendation-oriented instruction data for tuning the LLMs. Instruction tuning is key to elicit the abilities of LLMs for the recommendation tasks, and it further enables the generalization of LLMs on diverse user needs or requests.”
• Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2023, methodology): “Scale Backbone FR LR CS ILC TIGER [37] N/A N/A P5 [22], [23] 220M T5 InstructRec [19] 3B Flan-T5 TALLRec [21] 7B LLaMA LC-Rec 7B LLaMA “ Yes/No ” [21] or perform reranking based on a small number of candidate items [19].”
• Hard Negative Sampling via Large Language Models for Recommendation (2025, unknown): “With the development of LLM fine-tuning techniques, recent work has primarily focused on fine-tuning LLMs using recommendation-related corpora and aligning LLMs with traditional recommendations (Lin et al., 2024; Li et al., 2023a; Bao et al., 2023).”