Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction

By Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, Kun Gai. Knowledge Discovery and Data Mining, 2019

332 citations recorded by the source corpus.

Click-through rate (CTR) prediction is critical for industrial applications such as recommender system and online advertising. Practically, it plays an important role for CTR modeling in these applications by mining user interest from rich historical behavior data. Driven by the development of deep learning, deep CTR models with ingeniously designed architecture for user interest modeling have been proposed, bringing remarkable improvement of model performance over offline metric. However, great efforts are needed to deploy these complex models to online serving system for realtime inference, facing massive traffic request. Things turn to be more difficult when it comes to long sequential user behavior data, as the system latency and storage cost increase approximately linearly with the length of user behavior sequence. In this paper, we face directly the challenge of long sequential user behavior modeling and introduce our hands-on practice with the co-design of machine learning algorithm and online serving system for CTR prediction task. (i) From serving system view, we decouple the most resource-consuming part of user interest modeling from the entire model by designing a separate module named UIC (User Interest Center). UIC maintains the latest interest state for each user, whose update depends on realtime user behavior trigger event, rather than on traffic request. Hence UIC is latency free for realtime CTR prediction. (ii) From machine learning algorithm view, we propose a novel memory-based architecture named MIMN (Multi-channel user Interest Memory Network) to capture user interests from long sequential behavior data, achieving superior performance over state-of-the-art models. MIMN is implemented in an incremental manner with UIC module. Theoretically, the co-design solution of UIC and MIMN enables us to handle the user interest modeling with unlimited length of sequential behavior data. Comparison between model performance and system efficiency proves the effectiveness of proposed solution. To our knowledge, this is one of the first industrial solutions that are capable of handling long sequential user behavior data with length scaling up to thousands. It now has been deployed in the display advertising system in Alibaba.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition, Information Systems

View Paper: https://www.semanticscholar.org/paper/4ff28221f1d734d65ff78587219f3c2fc17c4c43

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Chang Zhou
2. Jie Tang
3. Yong Li

In Computer Vision and Pattern Recognition

1. Kun Gai
2. Chang Zhou
3. Jie Tang
4. Yu Zheng
5. Maxim Naumov
6. James Y. Zou
7. Xiaoqiang Zhu
8. Jianwei Zhang

In Information Systems

1. Yu Zheng
2. Chang Zhou
3. James Y. Zou
4. Xiaoqiang Zhu
5. Kun Gai
6. Yanan Niu
7. Jianwei Zhang
8. Zhe Wang

Citations in the corpus

• Controllable Multi-Interest Framework for Recommendation (2020, background): “MIMN.”
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems (2019, background): “At present, the out-sized memory consumption of standard embedding layers in these recommendation systems is a major burden on the memory hierarchy – an embedding layer for a single recommendation engine can consume tens of gigabytes of space [41, 44].”
• Sequential Recommendation with Graph Neural Networks (2021, methodology): “This dataset is widely used for recommendation research [20, 47], which is collected from the largest e-commerce platform in China.”
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction (2020, methodology): “A memory-based architecture named MIMN is proposed in [8], which embedded user long-term interest into fixed-sized memory network to solve the problem of large storage of user behavior data.”