Representation Learning with Large Language Models for Recommendation

By Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, Chao Huang. The Web Conference, 2023

427 citations recorded by the source corpus.

Recommender systems have seen significant advancements with the influence of deep learning and graph neural networks, particularly in capturing complex user-item relationships. However, these graph-based recommenders heavily depend on ID-based data, potentially disregarding valuable textual information associated with users and items, resulting in less informative learned representations. Moreover, the utilization of implicit feedback data introduces potential noise and bias, posing challenges for the effectiveness of user preference learning. While the integration of large language models (LLMs) into traditional ID-based recommenders has gained attention, challenges such as scalability issues, limitations in text-only reliance, and prompt input constraints need to be addressed for effective implementation in practical recommender systems. To address these challenges, we propose a model-agnostic framework RLMRec that aims to enhance existing recommenders with LLM-empowered representation learning. It proposes a recommendation paradigm that integrates representation learning with LLMs to capture intricate semantic aspects of user behaviors and preferences. RLMRec incorporates auxiliary textual signals, employs LLMs for user/item profiling, and aligns the semantic space of LLMs with collaborative relational signals through cross-view alignment. This work further demonstrates the theoretical foundation of incorporating textual signals through mutual information maximization, which improves the quality of representations. Our evaluation integrates RLMRec with state-of-the-art recommender models, while also analyzing its efficiency and robustness to noise data. Implementation codes are available at https://github.com/HKUDS/RLMRec.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/4829b73a47be18f73e9e8d90f3c23c8f84d0fccb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wayne Xin Zhao
2. Yupeng Hou
3. Hongyu Lu
4. Ruobing Xie
5. Junjie Zhang
6. Zihan Lin
7. Chao Huang
8. Qinyong Wang

In Information Systems

1. Yupeng Hou
2. Wayne Xin Zhao
3. Hongyu Lu
4. Chao Huang
5. Zihan Lin
6. Ruobing Xie
7. Junjie Zhang
8. Qinyong Wang

Citations in the corpus

• LLMRec: Large Language Models with Graph Augmentation for Recommendation (2023, background): “Future directions for investigation include integrating causal inference into side information debiasing and exploring counterfactual factors for context-aware user preference.”
• Large Language Models are Zero-Shot Rankers for Recommender Systems (2023, methodology): “We follow the experimental setting in Section 3.1 and replace the base LLM with LLaMA of different sizes (7B, 13B, 35B, and 65B) to investigate the effect of model scaling on the zero-shot recommendation task.”
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, neutral): “That is, the LLM is instructed to rerank the retrieved candidate items, enabling more accurate recommendations.”
• Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders (2024, neutral): “Recently, the emergence of LLMs with superior generalization capabilities has inspired researchers to explore us-ing LLMs as semantic encoders in recommendation (Wei et al., 2024; Ren et al., 2024; Ren and Huang, 2024; Sheng et al., 2024).”
• How Can Recommender Systems Benefit from Large Language Models: A Survey (2023, neutral): “In this way, LLM can provide aligned in-domain knowledge to CRM, while CRM act as a frequently updated adapter for LLM.”
• A survey on large language models for recommendation (2023, neutral): “Since under the fine-tuning paradigm, the utilization and training methodologies of generative LLMs are fundamentally similar to the discriminative LLMs discussed in Section 3.1, therefore, we will only introduce a few representative works in this subsection.”
• Hard Negative Sampling via Large Language Models for Recommendation (2025, unknown): “• RLMRec (Ren et al., 2024): This model constructs richer user/item embeddings by integrating LLM-generated semantic texts, improving generalization and robustness of downstream CF models.”