Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features

By Ying Shan, T. R. Hoens, Jian Jiao, Haijing Wang, Dong Yu, Jian Chang Mao. 2016

456 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition, Hardware and Architecture

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Xiangnan He
2. Tat-seng Chua
3. Xiang Wang
4. Meng Wang
5. Xing Xie
6. Liqiang Nie
7. Hanwang Zhang
8. Fei Wu

In Computer Vision and Pattern Recognition

1. Xiangnan He
2. Tat-seng Chua
3. Xiang Wang
4. Liqiang Nie
5. Fuli Feng
6. Meng Wang
7. Yi Tay
8. Hanwang Zhang

In Hardware and Architecture

1. Xiangnan He
2. Tat-seng Chua
3. Yi Tay
4. Jongsoo Park

Citations in the corpus

• Deep Interest Network for Click-Through Rate Prediction (2017, methodology): “approach to extract users’ interests from the rich historical behavior while building the click-through rate (CTR) prediction model. 4.1 Base Model Following the popular model structure introduced in [1, 4, 2], our base model is composed with two steps: i) transfer each sparse id feature into a embe”
• An Introduction to Neural Information Retrieval (2018, unknown): “These features, learnt from the query and document texts, can be combined with other features that may not be possible to infer from the content, such as document popularity [Shan et al., 2016].”
• KGAT: Knowledge Graph Attention Network for Recommendation (2019, methodology): “feature vector, together with user ID and item ID, and feed them into a supervised learning (SL) model to predict the score. Such a SL paradigm for recommendation has been widely deployed in industry [7, 24, 41], and some representative models include factorization machine (FM) [23], NFM (neural FM)”
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks (2018, methodology): “The embedding layer, which is a shared component in neural network-based methods [11, 19, 32], contains nd parameters, where n is the dimension of sparse representation of input feature and d is the embedding size.”
• A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, methodology): “Others injected specific structures in MLPs for better capturing feature interactions.”
• Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems (2020, background): “For example, Deep Crossing (Shan et al., 2016), Wide&Deep Learning (Cheng et al., 2016), YouTube Rec-ommendation CTR model (Covington et al., 2016) and Deep Interest Network (DIN) (Zhou et al., 2018) design special fully-connected layers for corresponding tasks to capture the latent interactions…”
• TEM: Tree-enhanced Embedding Model for Explainable Recommendation (2018, methodology): “Specially, Wide&Deep [12] and Deep Crossing [36] learn feature interactions by placing a multi-layer perceptron (MLP) above the concatenation of the embeddings of nonzero features; the MLP is claimed to be capable of learning anyorder cross features.”
• Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks (2017, methodology): “, 2016] and DeepCross [Shan et al., 2016] with a much simpler structure and fewer model parameters.”
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems (2020, background): “• It’s better to insert the cross layers in between the input and the hidden layers of DNN (also observed in [44]).”
• DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction (2022, background): “In CTR prediction tasks, the feature inputs usually contain discrete categorical terms (sparse features) and numerical values (dense features) [8, 11, 28, 35].”
• Deep Learning Based Recommender System (2017, methodology): “Deep neural network is a promising tool for feature cra‰ing without manual intervention [90].”
• Deep & Cross Network for Ad Click Predictions (2017, background): “We compare DCN with five models: the DCN model with no cross network (DNN), logistic regression (LR), Factorization Machines (FMs), Wide and Deep Model (W&D), and Deep Crossing (DC).”
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems (2018, methodology): “an open question. In this paper, we propose a neural network-based model to learn feature interactions in an explicit, vector-wise fashion. Our approach is based on the Deep & Cross Network (DCN) [36], which tries to efficiently capture effective feature interactions of bounded degrees and learn”
• TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval (2020, methodology): “Inspired by [27], where residual layers were used in a non-convolutional network in the NLP domain, they are adopted here to overcome over-€Šing and gradient vanishing problems.”