Behavior sequence transformer for e-commerce recommendation in Alibaba

By Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, Wenwu Ou. 2019

507 citations recorded by the source corpus.

Deep learning based methods have been widely used in industrial recommendation systems (RSs). Previous works adopt an Embedding&MLP paradigm: raw features are embedded into low-dimensional vectors, which are then fed on to MLP for final recommendations. However, most of these works just concatenate different features, ignoring the sequential nature of users' behaviors. In this paper, we propose to use the powerful Transformer model to capture the sequential signals underlying users' behavior sequences for recommendation in Alibaba. Experimental results demonstrate the superiority of the proposed model, which is then deployed online at Taobao and obtain significant improvements in online Click-Through-Rate (CTR) comparing to two baselines.

Subfields: Computer Vision and Pattern Recognition, Information Systems, Management Science and Operations Research, Large Language Models, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Trevor Darrell
4. Xiaogang Wang
5. Quoc V. Le
6. Ming-Hsuan Yang
7. Alan L. Yuille
8. Andrew Zisserman

In Information Systems

1. Kaigui Bian
2. Maksims Volkovs
3. Yuanxing Zhang
4. Guihai Chen
5. Wei Yan
6. Barret Zoph
7. Jimmy Ba
8. Pengyu Zhao

In Management Science and Operations Research

1. William Fedus
2. Barret Zoph
3. Joseph Dean
4. Kaigui Bian
5. Yuanxing Zhang
6. Pengyu Zhao
7. Shaojie Tang
8. Maksims Volkovs

In Large Language Models

1. Joseph Dean
2. William Fedus
3. Barret Zoph
4. Maksims Volkovs
5. Jimmy Ba
6. Yi Tay
7. Mostafa Dehghani
8. Donald Metzler

In Recommender Systems

1. Kaigui Bian
2. Maksims Volkovs
3. Yuanxing Zhang
4. Wei Yan
5. Pengyu Zhao
6. Kecheng Xiao

Citations in the corpus

• A Review of Sparse Expert Models in Deep Learning (2022, background): “Some examples are image classification (Dosovitskiy et al., 2020), object detection (Carion et al., 2020), recommendation systems (Chen et al., 2019), speech recognition (Dong et al., 2018; Nakatani, 2019; Gulati et al., 2020).”
• Secure Federated Submodel Learning (2019, methodology): “To improve accuracy, Deep Interest Network (DIN) [4] introduces the attention mechanism to activate the user’s historical behaviors, namely relative interests, with respect to the target item; Deep Interest Evolution Network (DIEN) [5] further extracts latent interests and monitors interest evolutio”
• Improving Transformer Optimization Through Better Initialization (2020, background): “…et al., 2017; Edunov et al., 2018), language modeling (Devlin et al., 2019; Yang et al., 2019; Lan et al., 2020; Beltagy et al., 2020), video cap-tioning and summarization (Bilkhu et al., 2019; Fan et al., 2018; Chen et al., 2018a) and recommender systems (Chen et al., 2019; Sun et al., 2019).”
• AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System (2020, methodology): “BERT4REC is a successful transplant of Transformer to recommendation, which is on behalf of Transformer-based recommender systems including SASREC [32], MBTREC[81], BST [6], etc.”