Deep & Cross Network for Ad Click Predictions

By Ruoxi Wang, Bin Fu, Gang Fu, Mingliang Wang. ADKDD@KDD, 2017

1,535 citations recorded by the source corpus.

Feature engineering has been the key to the success of many prediction models. However, the process is nontrivial and often requires manual feature engineering or exhaustive searching. DNNs are able to automatically learn feature interactions; however, they generate all the interactions implicitly, and are not necessarily efficient in learning all types of cross features. In this paper, we propose the Deep & Cross Network (DCN) which keeps the benefits of a DNN model, and beyond that, it introduces a novel cross network that is more efficient in learning certain bounded-degree feature interactions. In particular, DCN explicitly applies feature crossing at each layer, requires no manual feature engineering, and adds negligible extra complexity to the DNN model. Our experimental results have demonstrated its superiority over the state-of-art algorithms on the CTR prediction dataset and dense classification dataset, in terms of both model accuracy and memory usage.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ca870d3dbfdddaef4a92e60f53d2077dbe0fdd0a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning (2020, background): “It should be noted that this is a quite signiﬁcant accuracy increase for Recommendation Systems [51].”
FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction (2019, methodology): “The shallow baseline models include LR(logistic regression)[17], FM[19, 20], FFM[9, 10], AFM[23], and the deep baseline models include FNN[25], DCN[22], DeepFM[4], XDeepFM[15].”
Deep Session Interest Network for Click-Through Rate Prediction (2019, background): “DeepFM [Guo et al. , 2017] learns the second-order crossover of features and DCN [ Wang et al. , 2017 ] applies a multi-layer residual structure to learn higher-order representation of features.”
TabTransformer: Tabular Data Modeling Using Contextual Embeddings (2020, background): “These include Deep & Cross Networks (Wang et al. 2017), Wide & Deep Networks (Cheng et al.”
AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction (2020, methodology): “In most CTR prediction tasks, data is collected in a multi-field categorical form 1 [27, 32, 36].”
AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction (2020, background): “Recently, deep learning models have achieved state-of-the-art performance on some public benchmarks [16, 25].”
Recommending what video to watch next: a multitask ranking system (2019, background): “Many model architectures are designed to capture one type of information, such as feature crosses [39] or sequential information [35].”
Revisiting Deep Learning Models for Tabular Data (2021, methodology): “For these reasons, a large number of DL solutions were recently proposed, and new models continue to emerge (Arik and Pfister, 2020; Badirli et al., 2020; Hazimeh et al., 2020; Huang et al., 2020a; Klambauer et al., 2017; Popov et al., 2020; Song et al., 2019; Wang et al., 2017, 2020a).”
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems (2020, methodology): “This has significantly improved model performance for search and recommendation systems across the board [6, 13, 50].”
On Embeddings for Numerical Features in Tabular Deep Learning (2022, neutral): “During several recent years, the community has proposed a large number of deep models for tabular data [2, 3, 13, 15, 17, 21, 24, 31, 39, 40, 46].”
DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction (2022, methodology): “Deep Cross Net (DCN) is a widely used feature interaction Module in CTR prediction tasks [39].”
Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction (2020, methodology): “As shown in Figure 1, for a given instance, we assume its raw input dense features are concatenated into a vector, and its sparse features are embedded into low-dimensional vectors based on the look-up-table operation following similar preprocessing done in various CTR models [21, 30].”
Behavior sequence transformer for e-commerce recommendation in Alibaba (2019, background): “, DeepFM [6], XDeepFM [9], Deep and Cross networks [16], etc.”
Federated learning for predicting clinical outcomes in patients with COVID-19 (2021, methodology): “9a, the model fuses information from both EMR and CXR features (based on a modified ResNet34 with spatial attention 66 pretrained on the CheXpert dataset) 67 and the Deep & Cross network 68 .”
Federated Meta-Learning for Recommendation (2018, background): “Deep learning for recommender systems has been studied in the literature, from the perspective of pure deep neural networks (Cheng et al., 2016; Wang et al., 2017) or combination with other methods (He et al., 2017; Guo et al., 2017).”
AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System (2020, methodology): “DeepFM [18], DCN [70] and xDeepFM [39] enumerate interactions of bounded degrees and combine the feature interactions with MLP via shortcut connection while IPNN [51] and FibiNet [31] directly apply interactions to MLP.”
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction (2020, methodology): “ished SOTA industry solution for this task. 2 RELATED WORK UserInterestModel.Deep learning based methods have achieved great success in CTR prediction task[1, 11, 18]. In early age, most pioneer works[1, 4, 7, 9, 15] use a deep neural network to capture interactions between features from different f”
Controllable Multi-Interest Framework for Recommendation (2020, background): “DCN [53] keeps the benefits of a deep model and introduces a novel cross network that is more efficient in learning specific bounded-degree feature interactions.”
Conditional Wasserstein GAN-based Oversampling of Tabular Data for Imbalanced Learning (2020, methodology): “In order to evaluate the performance of our proposed cWGAN-based oversampling method, we compare our method against four state-of-the-art oversampling methods, Random Oversampling and no oversampling on seven real-world credit scoring datasets.”