xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

By Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, Guangzhong Sun. 2018

1,240 citations recorded by the source corpus.

Combinatorial features are essential for the success of many commercial models. Manually crafting these features usually comes with high cost due to the variety, volume and velocity of raw data in web-scale systems. Factorization based models, which measure interactions in terms of vector product, can learn patterns of combinatorial features automatically and generalize to unseen features as well. With the great success of deep neural networks (DNNs) in various fields, recently researchers have proposed several DNN-based factorization model to learn both low- and high-order feature interactions. Despite the powerful ability of learning an arbitrary function from data, plain DNNs generate feature interactions implicitly and at the bit-wise level. In this paper, we propose a novel Compressed Interaction Network (CIN), which aims to generate feature interactions in an explicit fashion and at the vector-wise level. We show that the CIN share some functionalities with convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We further combine a CIN and a classical DNN into one unified model, and named this new model eXtreme Deep Factorization Machine (xDeepFM). On one hand, the xDeepFM is able to learn certain bounded-degree feature interactions explicitly; on the other hand, it can learn arbitrary low- and high-order feature interactions implicitly. We conduct comprehensive experiments on three real-world datasets. Our results demonstrate that xDeepFM outperforms state-of-the-art models. We have released the source code of xDeepFM at https://github.com/Leavingseason/xDeepFM.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Wayne Xin Zhao
4. Tat-seng Chua
5. Meng Wang
6. Ji-rong Wen
7. Yi Tay
8. Yong Li

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Meng Wang
4. Tat-seng Chua
5. Wayne Xin Zhao
6. Yong Li
7. Chen Gao
8. Le Wu

Citations in the corpus

• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems (2020, background): “, 200 400 in [26, 46]).”
• Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems (2020, methodology): “…among features; Product-based Neural Network (PNN) (Qu et al., 2016) employs a product layer to capture high-order feature interactions; DeepFM (Guo et al., 2017) and xDeepFM (Lian et al., 2018) use factorization machines (FM) to model both low-order and high-order feature correlations.”
• Deep Learning Recommendation Model for Personalization and Recommendation Systems (2019, methodology): “We show some of the operators used in DLRM in PyTorch [23] and Caffe2 [8] [3, 13, 27, 18, 28, 29] use similar underlying ideas to generate higher-order terms to handle sparse features.”
• FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction (2019, methodology): “The shallow baseline models include LR(logistic regression)[17], FM[19, 20], FFM[9, 10], AFM[23], and the deep baseline models include FNN[25], DCN[22], DeepFM[4], XDeepFM[15].”
• Recommendation system based on deep learning methods: a systematic review and new directions (2019, methodology): “Other authors explore some strategies such as introducing pooling layers (Guo et al. 2017; Lian and Chen 2018) to strengthen the ability to learn the most useful higher order interaction.”
• DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation (2020, background): “Recently, some deep learning based models have been proposed to tackle the CF problem [16], [27].”
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction (2020, background): “In early age, most pioneer works[1, 4, 7, 9, 15] use a deep neural network to capture interactions between features from different fields so that engineers could get rid of boring feature engineering works.”
• Behavior sequence transformer for e-commerce recommendation in Alibaba (2019, background): “, DeepFM [6], XDeepFM [9], Deep and Cross networks [16], etc.”
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “Since the amount of input items in this stage is relatively small, the system can afford much more complicated algorithms to achieve higher recommendation accuracy [80, 101, 156].”
• A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, background): “applies cross interactions at the vector-wise level explicitly [65].”
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks (2018, methodology): “4266 CIN [19] 0.”
• RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms (2020, background): “xDeepFM [36] SIGKDD 2018 Criteo, Dianping, Bing News”
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction (2020, background): “ture interaction to control whether its output should be passed to the next layer. In previous works, the status of the gates is either specified beforehand by expert knowledge [6] or set as all open [8, 18]. From a data-driven point of view, whether open or closed a gate should depend on the contri”
• Deep Learning Based Recommender System (2017, methodology): “[93] improved DeepMF by proposing an eXtreme deep factorization machine to jointly model the explicit and implicit feature interactions.”
• Hierarchical Gating Networks for Sequential Recommendation (2019, background): “In [5, 21, 41], conventional matrix factorization and factorization machine methods are also benefited by the representation ability of deep neural networks.”