Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)

By Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, Yongfeng Zhang. ACM Conference on Recommender Systems, 2022

915 citations recorded by the source corpus.

For a long time, different recommendation tasks require designing task-specific architectures and training objectives. As a result, it is hard to transfer the knowledge and representations from one task to another, thus restricting the generalization ability of existing recommendation approaches. To deal with such issues, considering that language can describe almost anything and language grounding is a powerful medium to represent various problems or tasks, we present a flexible and unified text-to-text paradigm called “Pretrain, Personalized Prompt, and Predict Paradigm” (P5) for recommendation, which unifies various recommendation tasks in a shared framework. In P5, all data such as user-item interactions, user descriptions, item metadata, and user reviews are converted to a common format — natural language sequences. The rich information from natural language assists P5 to capture deeper semantics for personalization and recommendation. Specifically, P5 learns different tasks with the same language modeling objective during pretraining. Thus, it serves as the foundation model for various downstream recommendation tasks, allows easy integration with other modalities, and enables instruction-based recommendation. P5 advances recommender systems from shallow model to deep model to big model, and will revolutionize the technical form of recommender systems towards universal recommendation engine. With adaptive personalized prompt for different users, P5 is able to make predictions in a zero-shot or few-shot manner and largely reduces the necessity for extensive fine-tuning. On several benchmarks, we conduct experiments to show the effectiveness of P5. To help advance future research on Recommendation as Language Processing (RLP), Personalized Foundation Models (PFM), and Universal Recommendation Engine (URE), we release the source code, dataset, prompts, and pretrained P5 model at https://github.com/jeykigung/P5.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/df602516e28a9ef0ef665ed0aef551984d8d770d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wayne Xin Zhao
2. Ji-rong Wen
3. Hongzhi Yin
4. Tong Chen
5. Yupeng Hou
6. Xin Xia
7. Junliang Yu
8. Hongyu Lu

In Information Systems

1. Hongzhi Yin
2. Wayne Xin Zhao
3. Tong Chen
4. Yupeng Hou
5. Ji-rong Wen
6. Junliang Yu
7. Xin Xia
8. Hongyu Lu

Citations in the corpus

• Is ChatGPT a Good Recommender? A Preliminary Study (2023, methodology): “We randomly sample 100 records from the test set proposed by P5 [19] for evaluation.”
• Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System (2023, methodology): “Applying LLMs for addressing the recommendation tasks has received several preliminary research experimental trials already [12,7,25].”
• Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2023, methodology): “• P5-CID [22], [23] organizes multiple recommendation tasks in a text-to-text format and models different tasks uniformly using the T5 model.”
• Language is All a Graph Needs (2023, background): “P5 (Geng et al., 2022; Hua et al., 2023; Xu et al., 2023) and its variants (Geng et al., 2023; Hua et al., 2024; Ji et al., 2024), further contributes to this trend by reformulating all personalized recommendation tasks as language modeling tasks via prompts.”
• Representation Learning with Large Language Models for Recommendation (2023, methodology): “In recent times, there have been several endeavors to leverage diverse data modalities in order to enhance traditional ID-based recommenders [9, 17, 46].”
• Recommender Systems with Generative Retrieval (2023, methodology): “9, 48], app [2], product [6, 8], and music [18, 19].”
• When large language models meet personalization: perspectives of challenges and opportunities (2023, neutral): “However, there still exists a substantial gap when compared to traditional recommendation models, particularly fine-tuned large language models designed specifically for recommenders, such as P5 [174] and M6-Rec [175].”
• Recommender Systems in the Era of Large Language Models (LLMs) (2023, neutral): “Details of the two recommender systems are illustrated in Figure 2.”
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, positive): “, T5 [29] and GPT-3 [4]) have shown remarkable abilities on a variety of natural language tasks, which also shed lights on developing more general and effective recommender systems [2, 6, 7, 13, 17, 18].”
• Self-Supervised Learning for Recommender Systems: A Survey (2022, methodology): “A very recent work P5 [116] takes the first step to explore the combination of prompt learning [53] and personalization.”
• Large Language Models are Zero-Shot Rankers for Recommender Systems (2023, methodology): “The results demonstrate that instruction tuning improves the ranking abilities of LLMs, even if the instructions are not specially designed for the recommendation tasks.”
• OpenAGI: When LLM Meets Domain Experts (2023, background): “This results in superior performance across diverse natural language processing tasks [2, 36].”