S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization

By Kun Zhou, Haibo Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, Ji-rong Wen. International Conference on Information and Knowledge Management, 2020

1,154 citations recorded by the source corpus.

Recently, significant progress has been made in sequential recommendation with deep learning. Existing neural sequential recommendation models usually rely on the item prediction loss to learn model parameters or data representations. However, the model trained with this loss is prone to suffer from data sparsity problem. Since it overemphasizes the final performance, the association or fusion between context data and sequence data has not been well captured and utilized for sequential recommendation. To tackle this problem, we propose the model S3-Rec, which stands for Self-Supervised learning for Sequential Recommendation, based on the self-attentive neural architecture. The main idea of our approach is to utilize the intrinsic data correlation to derive self-supervision signals and enhance the data representations via pre-training methods for improving sequential recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence by utilizing the mutual information maximization (MIM) principle. MIM provides a unified way to characterize the correlation between different types of data, which is particularly suitable in our scenario. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods, especially when only limited training data is available. Besides, we extend our self-supervised learning method to other recommendation models, which also improve their performance.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/39dbb388062b63e82e984d46e2c20b5ca9885b42

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Wayne Xin Zhao
3. Hongzhi Yin
4. Xiang Wang
5. Ji-rong Wen
6. Junliang Yu
7. Yongfeng Zhang
8. Tong Chen

In Information Systems

1. Xiangnan He
2. Hongzhi Yin
3. Xiang Wang
4. Wayne Xin Zhao
5. Junliang Yu
6. Meng Wang
7. Tong Chen
8. Xin Xia

Citations in the corpus

• Is ChatGPT a Good Recommender? A Preliminary Study (2023, methodology): “And we adopt top-k Hit Ratio (HR@ k ), top-k Normalized Discounted Cumulative Gain (NDCG@ k ) for sequential recommendation and direct recommendation which are widely used in re-lated works [19, 68].”
• Self-supervised Graph Learning for Recommendation (2020, neutral): “A very recent one is S3Rec [49] for sequential recommendation which utilizes the mutual information maximization principle to learn the correlations among attribute, item, subsequence, and sequence.”
• Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation (2021, background): “These efforts either mine self-supervision signals from future/surrounding sequential data [24, 45], or mask attributes of items/users to learn correlations of the raw data [64].”
• Self-supervised Learning for Large-scale Item Recommendations (2020, background): “More recently, a line of research studies self-supervised learning improving sequential user modeling in recommendations [27, 37, 43].”
• Recommender Systems with Generative Retrieval (2023, methodology): “For this, we propose a generative approach that directly predicts the Semantic IDs of items.”
• Large Language Models are Zero-Shot Rankers for Recommender Systems (2023, background): “…the real user preference, since existing models mainly capture user preference from historical interaction behaviors, e.g., clicked item sequences [14, 19, 39, 17], limiting the expressive power to model the complicated user interests ( e.g., user intentions expressed in natural language).”
• Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation (2020, background): “As self-supervised learning is still in its infancy, there are only several studies combining it with recommender systems (Zhou et al. 2020; Ma et al. 2020; Xin et al. 2020).”
• Filter-enhanced MLP is All You Need for Sequential Recommendation (2022, methodology): “Following [68], we employ top-k Hit Ratio (HR@k), top-k Normalized Discounted Cumulative Gain (NDCG@k), andMean Reciprocal Rank (MRR) for evaluation, which are widely used in related works [22, 42, 70].”
• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022, background): “Besides classical rating prediction and direct user-item matching-based recommendation for users, recent works are broadening the spectrum to new application scenarios such as sequential recommendation [20, 56, 59, 76], conversational recommendation [7, 57, 72], explainable recommendation [29, 58, 6”
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, neutral): “This involved the random sampling of 99 negative items and the target item, resulting in a collection of 100 candidate items.”
• RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms (2020, neutral): “S3Rec [92] CIKM 2020 Meituan, Amazon, Yelp, LastFM”
• A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, methodology): “…[233],SSE-PT [234] RNN based models RRN [235],BINN [204],HIERNN [236] Attention based models for user interest modeling CTRec [237],M3 [238],S3-rec [239],CTA [240], MTAM [241],TASER [242],ReChorus [243] Memory Networks for long distance item correlations KA-MemNN [244],CSRM [245],MTAM [241] CNN…”
• Towards Universal Sequence Representation Learning for Recommender Systems (2022, methodology): “To adapt to a new domain, previous pre-training based recommendation methods [4, 36] usually require fine-tuning the whole network architecture, which is time-consuming and less flexible.”
• XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation (2022, background): “To the best of our knowledge, S 3 -Rec [15] is the first work that combines CL with sequential recommendation.”
• Self-Supervised Learning for Recommender Systems: A Survey (2022, methodology): “S 3 -Rec [26] applies item masking and item cropping to augment sequences and devises four contrastive tasks for pre-training a bidirectional Transformer for next-item prediction: item-attribute mutual information maximization (MIM), sequence-item MIM, sequence-attribute MIM, and sequence-sequence…”
• Contrastive Learning for Sequential Recommendation (2022, methodology): “• S -Rec MIP [50].”
• Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2023, methodology): “Following previous works [2], [10], [53], we employ the leave-one-out strategy for evaluation.”
• Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation (2021, methodology): “An increasing number of very recent studies [28, 32, 38, 40, 44, 45] have sought to harness CL for improving recommendation performance and have demonstrated significant gains.”