Feature-level Deeper Self-Attention Network for Sequential Recommendation

By Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, Xiaofang Zhou. International Joint Conference on Artificial Intelligence, 2019

458 citations recorded by the source corpus.

Sequential recommendation, which aims to recommend next item that the user will likely interact in a near future, has become essential in various Internet applications. Existing methods usually consider the transition patterns between items, but ignore the transition patterns between features of items. We argue that only the item-level sequences cannot reveal the full sequential patterns, while explicit and implicit feature-level sequences can help extract the full sequential patterns. In this paper, we propose a novel method named Feature-level Deeper Self-Attention Network (FDSA) for sequential recommendation. Specifically, FDSA first integrates various heterogeneous features of items into feature sequences with different weights through a vanilla mechanism. After that, FDSA applies separated self-attention blocks on item-level sequences and feature-level sequences, respectively, to model item transition patterns and feature transition patterns. Then, we integrate the outputs of these two blocks to a fully-connected layer for next item recommendation. Finally, comprehensive experimental results demonstrate that considering the transition relationships between features can significantly improve the performance of sequential recommendation.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/63555b4f363ee72dd8d49d8deca68760542c0cd2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wayne Xin Zhao
2. Xiangnan He
3. Ji-rong Wen
4. Yongfeng Zhang
5. Yupeng Hou
6. Yi Tay
7. Fuli Feng
8. Julian McAuley

In Information Systems

1. Xiangnan He
2. Wayne Xin Zhao
3. Yongfeng Zhang
4. Yupeng Hou
5. Ji-rong Wen
6. Fuli Feng
7. Xiang Wang
8. Yi Tay

Citations in the corpus

• Text Is All You Need: Learning Language Representations for Sequential Recommendation (2023, methodology): “Unlike previous sequential recommenders [12, 37] using both text and item IDs, in this study, we use only text for the sequential recommendation.”
• Is ChatGPT a Good Recommender? A Preliminary Study (2023, methodology): “For sequential recommendation, we adopt Caser [50], HGN [34], GRU4Rec [21], BERT4Rec [48], FDSA [63], SASRec [24] and S3-Rec [67] as baselines for comparison.”
• Recommender Systems with Generative Retrieval (2023, methodology): “• FDSA [47]: Feature-level Deeper Self-Attention Network (FDSA) incorporates item features in addition to the item embeddings as part of the input sequence in the Transformers.”
• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022, methodology): “FDSA [69] focuses on the feature transition patterns by modeling feature sequence with a self-attention module.”
• Towards Universal Sequence Representation Learning for Recommender Systems (2022, methodology): “recommendation methods ( i.e., FDSA and S -Rec) perform better than the i.e., 𝑡 + 𝐼𝐷 𝑡 + 𝐼𝐷 i.e., Table 3: Performance comparison of different recommendation models.”
• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation (2023, methodology): “Moving forward, we plan to explore more efficient methods to activate the recommendation ability of larger models and tune LLMs to handle multiple recommendation tasks simultaneously.”
• S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization (2020, methodology): “a similar effect to capture sequential dependencies as in [8, 21] except that it can also utilize bidirectional sequential information. Attribute-aware sequential models such as TransFM [16] and FDSA [29] leverage the contextual features to improve the sequential recommender models, in which these f”
• Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2023, background): “• FDSA [9] focuses on the transformation patterns between item features, modeling both item-level and feature-level sequences separately through self-attention networks.”
• RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms (2020, unknown): “SASRecF(+feature embedding) [84] IJCAI 2019 -”