ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation

By Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, Jun Gao. 2017

333 citations recorded by the source corpus.

A user can be represented as what he/she does along the history. A common way to deal with the user modeling problem is to manually extract all kinds of aggregated features over the heterogeneous behaviors, which may fail to fully represent the data itself due to limited human instinct. Recent works usually use RNN-based methods to give an overall embedding of a behavior sequence, which then could be exploited by the downstream applications. However, this can only preserve very limited information, or aggregated memories of a person. When a downstream application requires to facilitate the modeled user features, it may lose the integrity of the specific highly correlated behavior of the user, and introduce noises derived from unrelated behaviors. This paper proposes an attention based user behavior modeling framework called ATRank, which we mainly use for recommendation tasks. Heterogeneous user behaviors are considered in our model that we project all types of behaviors into multiple latent semantic spaces, where influence can be made among the behaviors via self-attention. Downstream applications then can use the user behavior vectors via vanilla attention. Experiments show that ATRank can achieve better performance and faster training process. We further explore ATRank to use one unified model to predict different types of user behaviors at the same time, showing a comparable performance with the highly optimized individual models.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hongxia Yang
2. Chang Zhou
3. Wenwu Zhu
4. Jie Tang
5. Peng Cui
6. Yi Tay
7. Yun Raymond Fu
8. Shuai Zhang

In Information Systems

1. Yi Tay
2. Aixin Sun
3. Lina Yao
4. Shuai Zhang
5. Peng Cui
6. Wenwu Zhu
7. Weichen Shen
8. Fei Sun

Citations in the corpus

• Controllable Multi-Interest Framework for Recommendation (2020, methodology): “n outbreaking method as tensor2tensor [51]. BERT [10] leverages tensor2tensor and achieves giant success in natural language processing. The attention mechanism is also adapted to recommender systems [6, 59] and is rather useful on real-world recommendation tasks. Capsule Network. The concept of “ca”
• AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System (2020, methodology): “, ATRank [82], SASRec [32], and BERT4Rec [63] introduce self-attention layers and point-wise feedforward layers to capture long-term semantics, which are the same as the stack of multi-head attention layers and 1× 1 convolutions in the choice blocks.”
• Deep Learning Based Recommender System (2017, methodology): “Attention-based CNNs are capable of capturing the most informative elements of the inputs [128].”
• Deep Session Interest Network for Click-Through Rate Prediction (2019, methodology): “Deep Interest Evolution Network (DIEN) [Zhou et al. , 2018b] uses auxiliary loss to adjust the expression of current behavior to the next behavior and then models the specific interest evolving process for different target items with AUGRU.”
• Explainable Interaction-driven User Modeling over Knowledge Graph for Sequential Recommendation (2019, methodology): “Recently, self-attention based sequential recommendation algorithms have attracted increasing attention due to the flexibility and efficiency of the model [12, 33].”
• Deep Multimodal Representation Learning: A Survey (2019, background): “Some other researches also demonstrated the promising perspective of attention based methods for multimodal features fusion [166], [167].”
• An Attentive Survey of Attention Models (2019, background): “, 2018; Shuai Yu, 2019], or within an encoderdecoder architecture for sequential recommendations [Kang and McAuley, 2018; Zhou et al., 2018].”
• Learning Disentangled Representations for Recommendation (2019, background): “Despite their notable success, the existing user behavior-based representation learning methods, such as the recent deep approaches [49, 32, 31, 52, 11, 18], generally neglect the complex interaction among the latent factors behind the users’ decision making processes.”
• Log2Intent: Towards Interpretable User Modeling via Recurrent Semantics Memory Unit (2019, background): “User modeling [11, 27, 40] enables personalized experience of software and web service, with a wide range of applications in recommender system, personalized social web, online advertising, email marketing, intelligent user interfaces, etc.”