Deep Interest Network for Click-Through Rate Prediction
By Guorui Zhou, Cheng-Ning Song, Xiaoqiang Zhu, Xiao Ma, Yanghui Yan, Xi-Wang Dai, Han Zhu, Junqi Jin, Han Li, Kun Gai. 2017
2,280 citations recorded by the source corpus.
Click-through rate prediction is an essential task in industrial applications, such as online advertising. Recently deep learning based models have been proposed, which follow a similar Embedding&MLP paradigm. In these methods large scale sparse input features are first mapped into low dimensional embedding vectors, and then transformed into fixed-length vectors in a group-wise manner, finally concatenated together to fed into a multilayer perceptron (MLP) to learn the nonlinear relations among features. In this way, user features are compressed into a fixed-length representation vector, in regardless of what candidate ads are. The use of fixed-length vector will be a bottleneck, which brings difficulty for Embedding&MLP methods to capture user's diverse interests effectively from rich historical behaviors. In this paper, we propose a novel model: Deep Interest Network (DIN) which tackles this challenge by designing a local activation unit to adaptively learn the representation of user interests from historical behaviors with respect to a certain ad. This representation vector varies over different ads, improving the expressive ability of model greatly. Besides, we develop two techniques: mini-batch aware regularization and data adaptive activation function which can help training industrial deep networks with hundreds of millions of parameters. Experiments on two public datasets as well as an Alibaba real production dataset with over 2 billion samples demonstrate the effectiveness of proposed approaches, which achieve superior performance compared with state-of-the-art methods. DIN now has been successfully deployed in the online display advertising system in Alibaba, serving the main traffic.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Recommender Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Xiangnan He
- Quoc V. Le
- Tat-seng Chua
- Mohit Bansal
- Yoshua Bengio
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Yi Yang
- Ming-Hsuan Yang
- Alan L. Yuille
- Jitendra Malik
In Recommender Systems
Citations in the corpus
- Knowledge Graph Convolutional Networks for Recommender Systems (2019, background): “With the advance of Internet technology, people can access a vast amount of online content, such as news [25], movies [5], and commodities [26].”
- RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems (2018, background): “The idea was soon transplanted to recommender systems [4, 23, 33, 36, 47].”
- DKN: Deep Knowledge-Aware Network for News Recommendation (2018, methodology): “To characterize user’s diverse interests, we use an attention network [47, 54] to model the different impacts of the user’s clicked news on the candidate news.”
- Secure Federated Submodel Learning (2019, methodology): “Except the embedding layer for user IDs, goods IDs, and category IDs, the parameters of the other network layers in DIN, including the attention layer and the fully connected layer, are of size 64,327.”
- AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations (2020, background): “Compared to hand-crafted features, well-learned embeddings have been demonstrated to significantly enhance the recommendation performance [5, 8, 21, 24, 38].”
- MLPerf Training Benchmark (2019, background): “Recommendation systems are a key commercial workload for internet companies (Naumov et al., 2019; Zhou et al., 2018; Cheng et al., 2016).”
- Cross-domain Recommendation Without Sharing User-relevant Data (2019, neutral): “Besides, some works utilized neural networks to extract the auxiliary information and features in recommender systems, such as textual [25, 47], visual [3, 38], video [5], and hybrid [48].”
- DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference (2020, background): “For compute intensive models (e.g., DLRM-RMC3, WnD), system throughput is optimized at lower batch sizes compared to memory intensive models (e.g., DLRM-RMC1, DIN).”
- RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing (2019, neutral): “experience [21], [25], [29], [44], [49].”
- A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “In addition, in e-commerce platform, products may have rich attributes, such as price [244], category [248], etc., based on which heterogeneous graphs can be constructed [117].”
- xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems (2018, methodology): “[45] shows tha traditional RSs can not capture interest diversity and local activation effectively, so they introduce a Deep Interest Network (DIN) to represent users’ diverse interests with an attentive activation mechanism.”
- Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale (2021, background): “The advancement has led to state-of-the-art production models that are complex, compute intensive and consume large amount of memory resources [17, 27, 40, 60, 65, 66].”
- RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms (2020, neutral): “DIN [91] SIGKDD 2018 Amazon, MovieLens, Alibaba”
- Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction (2020, methodology): “• DIN [20] is an early work for user behavior modeling which proposes to soft-search user behaviors w.”
- Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate (2018, methodology): “Most of them follow a similar Embedding&MLP network architecture, as introduced in [3].”
- Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System (2020, background): “Personalized recommendation has revolutionized a myriad of online applications such as e-commerce [48, 50], search engines [38], and conversational systems [26, 40].”