Text Is All You Need: Learning Language Representations for Sequential Recommendation

By Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, Julian McAuley. Knowledge Discovery and Data Mining, 2023

383 citations recorded by the source corpus.

Sequential recommendation aims to model dynamic user behavior from historical interactions. Existing methods rely on either explicit item IDs or general textual features for sequence modeling to understand user preferences. While promising, these approaches still struggle to model cold-start items or transfer knowledge to new datasets. In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets. To this end, we present a novel framework, named Recformer, which effectively learns language representations for sequential recommendation. Specifically, we propose to formulate an item as a "sentence" (word sequence) by flattening item key-value attributes described by text so that an item sequence for a user becomes a sequence of sentences. For recommendation, Recformer is trained to understand the "sentence" sequence and retrieve the next "sentence". To encode item sequences, we design a bi-directional Transformer similar to the model Longformer but with different embedding layers for sequential recommendation. For effective representation learning, we propose novel pretraining and finetuning methods which combine language understanding and recommendation tasks. Therefore, Recformer can effectively recommend the next item based on language representations. Extensive experiments conducted on six datasets demonstrate the effectiveness of Recformer for sequential recommendation, especially in low-resource and cold-start settings.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/c551ef5a45f72a689cbf02877290115606019106

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lianghao Xia
2. Vojtěch Vančura
3. Qinyong Wang
4. Chao Huang
5. Suqi Cheng
6. Xubin Ren
7. Pavel Kordík
8. Wei Wei

In Information Systems

1. Lianghao Xia
2. Qinyong Wang
3. Chao Huang
4. Suqi Cheng
5. Xubin Ren
6. Wei Wei
7. Lixin Su
8. Junfeng Wang

Citations in the corpus

• Representation Learning with Large Language Models for Recommendation (2023, background): “In recent times, there have been several endeavors to leverage diverse data modalities in order to enhance traditional ID-based recommenders [9, 17, 46].”
• LLMRec: Large Language Models with Graph Augmentation for Recommendation (2023, background): “Firstly , LLMRec embraces the shift from an ID-based recommendation framework to a modality-based paradigm [18, 46].”
• beeFormer: Bridging the Gap Between Semantic and Interaction Similarity in Recommender Systems (2024, background): “Promising approach is to create artificial text sequences combining descriptions of interacted items and train a Transformer-based model on them [21, 25].”