Deep Interest Evolution Network for Click-Through Rate Prediction

By Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, Kun Gai. 2018

1,336 citations recorded by the source corpus.

Click-through rate (CTR) prediction, whose goal is to estimate the probability of a user clicking on the item, has become one of the core tasks in the advertising system. For CTR prediction model, it is necessary to capture the latent user interest behind the user behavior data. Besides, considering the changing of the external environment and the internal cognition, user interest evolves over time dynamically. There are several CTR prediction methods for interest modeling, while most of them regard the representation of behavior as the interest directly, and lack specially modeling for latent interest behind the concrete behavior. Moreover, little work considers the changing trend of the interest. In this paper, we propose a novel model, named Deep Interest Evolution Network (DIEN), for CTR prediction. Specifically, we design interest extractor layer to capture temporal interests from history behavior sequence. At this layer, we introduce an auxiliary loss to supervise interest extracting at each step. As user interests are diverse, especially in the e-commerce system, we propose interest evolving layer to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the sequential structure novelly, and the effects of relative interests are strengthened during interest evolution. In the experiments on both public and industrial datasets, DIEN significantly outperforms the state-of-the-art solutions. Notably, DIEN has been deployed in the display advertisement system of Taobao, and obtained 20.7% improvement on CTR.

Subfields: Information Systems, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Sociology and Political Science

Citations in the corpus

Secure Federated Submodel Learning (2019, methodology): “To improve accuracy, Deep Interest Network (DIN) [4] introduces the attention mechanism to activate the user’s historical behaviors, namely relative interests, with respect to the target item; Deep Interest Evolution Network (DIEN) [5] further extracts latent interests and monitors interest evolutio”
AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System (2020, methodology): “• DIEN [83] enhances DIN by using GRU and AUGRU module to extract user’s evolving interests, where an auxiliary loss is employed as a regularization term to make hidden states more expressive.”
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction (2020, methodology): “Typically, attention-based CTR models, such as DIN[20] and DIEN[19], design complex model structure and involve attention mechanism to capture user diverse interest by searching effective knowledge from user behavior sequence, with inputs from different candidate items.”
DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, background): “The optimal threshold varies across the three recommendation models, DLRM-RMC3, DLRM-RMC1, and DIEN.”
AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction (2020, background): “The revenue of an online advertising system depends critically on the performance of CTR prediction [9, 26] because candidate advertisements are ranked by CTR × bid, where bid is the price to pay once the advertisement is clicked [9, 39, 40].”
RecShard: statistical feature-based memory optimization for industry-scale neural recommendation (2022, background): “A significant fraction of deep learning compute cycles in industry-scale data centers can be attributed to deep learning recommendation models (DLRMs) [3, 4, 9, 11, 20, 34, 43, 47, 48].”
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale (2020, methodology): “Alibaba’s advertisement recommendation models use GPUs, at least, for inference [59].”
Sequential Recommendation with Graph Neural Networks (2021, methodology): “When the two-layer GRU structure in DIEN models user interests, the next GRU input depends on the previous GRU output, making it easier to be disturbed by the noise in long sequences.”
Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems (2019, background): “A wide variety of models has been developed over the recent years for this task, including but not limited to [12, 20, 30, 36, 57, 58].”
Domain-specific Genetic Algorithm for Multi-tenant DNNAccelerator Scheduling (2021, neutral): “Lang ALBERT [59], Bart [62], BERT [24], CamemBERT [68], CTRL [52], DistilBERT [91], ELECTRA [18], FlauBERT [60], GPT2 [82], GPT [81], LongFormer [6], MarianNMT [49], MobileBERT [103], Reformer [53], RetriBERT [44], RoBERTa [64], T5 [83], TransformerXL [22], XLM [58], XLM-RoBERTa [19], XLNet [119] Re”
Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction (2019, methodology): “According to our hands-on experience, it costs a lot of engineering work to implement DIEN [30] in our system.”
Deep Session Interest Network for Click-Through Rate Prediction (2019, methodology): “Deep Interest Evolution Network (DIEN) [Zhou et al. , 2018b] uses auxiliary loss to adjust the expression of current behavior to the next behavior and then models the speciﬁc interest evolving process for different target items with AUGRU.”
STAN: Spatio-Temporal Attention Network for Next Location Recommendation (2021, background): “They have achieved satisfactory performances on variety of tasks, such as session-based recommendation [11, 16], next basket recommendation [37] and next item recommendation [1, 44].”
When large language models meet personalization: perspectives of challenges and opportunities (2023, positive): “Deep learning-based methods offer approaches to capture high-order interactions by employing techniques like attention mechanisms [17, 18] and graph-based networks [19] to mine complex relationships between user and item.”
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale (2021, background): “Recommender systems have evolved with increasing model complexities: from simple content and collaborative ﬁltering, to matrix factorization [37, 50], multi-layer perceptrons and deep learning models [18, 47, 54, 55, 64, 65, 67].”
Deep Learning Recommendation Model for Personalization and Recommendation Systems (2019, methodology): “We show some of the operators used in DLRM in PyTorch [23] and Caffe2 [8] [3, 13, 27, 18, 28, 29] use similar underlying ideas to generate higher-order terms to handle sparse features.”