Self-Attentive Sequential Recommendation

By Wang-Cheng Kang, Julian McAuley. Industrial Conference on Data Mining, 2018

3,858 citations recorded by the source corpus.

Sequential dynamics are a key feature of many modern recommender systems, which seek to capture the 'context' of users' activities on the basis of actions they have performed recently. To capture such patterns, two approaches have proliferated: Markov Chains (MCs) and Recurrent Neural Networks (RNNs). Markov Chains assume that a user's next action can be predicted on the basis of just their last (or last few) actions, while RNNs in principle allow for longer-term semantics to be uncovered. Generally speaking, MC-based methods perform best in extremely sparse datasets, where model parsimony is critical, while RNNs perform better in denser datasets where higher model complexity is affordable. The goal of our work is to balance these two goals, by proposing a self-attention based sequential model (SASRec) that allows us to capture long-term semantics (like an RNN), but, using an attention mechanism, makes its predictions based on relatively few actions (like an MC). At each time step, SASRec seeks to identify which items are 'relevant' from a user's action history, and use them to predict the next item. Extensive empirical studies show that our method outperforms various state-of-the-art sequential models (including MC/CNN/RNN-based approaches) on both sparse and dense datasets. Moreover, the model is an order of magnitude more efficient than comparable CNN/RNN-based models. Visualizations on attention weights also show how our model adaptively handles datasets with various density, and uncovers meaningful patterns in activity sequences.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems

View Paper: https://www.semanticscholar.org/paper/97faeefa771e8cc8e55159e2bd03e6f5eef249a8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

Citations in the corpus

Towards Universal Sequence Representation Learning for Recommender Systems (2022, methodology): “Despite the progress, most of existing SRL methods for recommendation rely on explicit item IDs for developing the sequence models [6, 10].”
Text Is All You Need: Learning Language Representations for Sequential Recommendation (2023, methodology): “We can see that methods with item text (i.e., UniSRec and Recformer) outperform ID-only method SASRec especially when less training data is available.”
Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation (2021, methodology): “SASRec [11] brings self-attention into SRSs and BERT4Rec [23] employs BERT model to learn bidirectional item dependencies for sequential recommendation.”
SVP-CF: Selection via Proxy for Collaborative Filtering Data (2021, background): “• Self-attentive Sequential Recommendation (SASRec) (Kang & McAuley, 2018): Another sequential recommendation algorithm that relies on the sequence modeling capabilities of self-attentive neural networks (Vaswani et al., 2017).”
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, methodology): “Some tools for sequence modeling have been used, such as Markov chains [27] or recurrent neural networks [80].”
ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation (2020, methodology): “Despite the recent success of various neural approaches [11, 16, 18, 20], they are developed in an offline manner, in which the recommender is trained on a very large static training set and evaluated on a very restrictive testing set in a one-time process.”
Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation (2020, methodology): “ [37] training approach or a masked objective [9]. Amongst these models, researchers have paid special attention to three lines of work: RNN-based [16], CNNbased [9, 32, 37], and pure attention-based [20] sequential models. Generally speaking, standard RNN-based models rely on strict sequential depe”
SSE-PT: Sequential Recommendation Via Personalized Transformer (2020, methodology): “The first 4 have exactly the same train/dev/test splits as in (Kang and McAuley, 2018).”
Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited (2023, methodology): “In this paper, we perform rigorous empirical studies on two most commonly adopted recommendation paradigms: DSSM [24] and SASRec [25].”
When large language models meet personalization: perspectives of challenges and opportunities (2023, positive): “Self-Attentions [20] and Gated Recurrent Units (GRUs) [21] are popular choices for modeling sequential data in recommender systems.”
Behavior sequence transformer for e-commerce recommendation in Alibaba (2019, methodology): “tried to capture different similarities between the previously clicked items and the target item. In other words, our model with transformer are more suitable for capturing the sequential signals. In [7, 12], the Transformer model is proposed to solve the sequential recommendation problem in sequenc”
Filter-enhanced MLP is All You Need for Sequential Recommendation (2022, methodology): “We conduct experiments on GRU4Rec [17] and SASRec [22], two representative sequential recommendation models.”
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer (2019, methodology): “For sequential recommendation, Kang and McAuley [22] introduce a two-layer Transformer decoder (i.”
Contrastive Learning for Sequential Recommendation (2022, methodology): “• SASRec [23].”