Wide & Deep Learning for Recommender Systems

By Heng-Tze Cheng, L. Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, H. Aradhye, G. Anderson, Greg S. Corrado, Wei Chai, M. Ispir and 6 more. 2016

4,198 citations recorded by the source corpus.

Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort. With less feature engineering, deep neural networks can generalize better to unseen feature combinations through low-dimensional dense embeddings learned for the sparse features. However, deep neural networks with embeddings can over-generalize and recommend less relevant items when the user-item interactions are sparse and high-rank. In this paper, we present Wide & Deep learning---jointly trained wide linear models and deep neural networks---to combine the benefits of memorization and generalization for recommender systems. We productionized and evaluated the system on Google Play, a commercial mobile app store with over one billion active users and over one million apps. Online experiment results show that Wide & Deep significantly increased app acquisitions compared with wide-only and deep-only models. We have also open-sourced our implementation in TensorFlow.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Representation & Self-Supervised Learning

Citations in the corpus

Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (2018, background): “Borrowing the idea from wide and deep network (Cheng et al., 2016), manual features have also been added to combine with the outputs of previous layer for a more comprehensive representation.”
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (2019, background): “, 2019), while sparse vectors are superior at encoding precise lexical information such as term frequencies (Cheng et al., 2016).”
TensorFlow: A system for large-scale machine learning (2016, neutral): ““Wide and deep learning” creates distributed representations from cross-product transformations on categorical features, and the implementation on TensorFlow is used to power the Google Play app store recommender system [12].”
Knowledge Graph Convolutional Networks for Recommender Systems (2019, background): “To address these limitations, researchers usually turn to feature-rich scenarios, where attributes of users and items are used to compensate for the sparsity and improve the performance of recommendation [3, 17].”
DRN: A Deep Reinforcement Learning Framework for News Recommendation (2018, methodology): “• W&D [8].”
RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems (2018, background): “• Wide&Deep [6] is a general deep model for recommendation combining a (wide) linear channel with a (deep) nonlinear channel.”
DKN: Deep Knowledge-Aware Network for News Recommendation (2018, methodology): “In addition to the aforementioned DSSM [16], DeepWide [6], DeepFM [13], YouTubeNet [8] and DMF [50], other popular deep-learning-based recommender systems include Collaborative Deep Learning [44], SHINE [45], Multi-view Deep Learning [11], and Neural Collaborative Filtering [14].”
Deep Interest Network for Click-Through Rate Prediction (2017, methodology): “Most of the popular model structures [3, 4, 21] share a similar Embedding&MLP paradigm, which we refer to as base model, as shown in the left of Fig.”
Secure Federated Submodel Learning (2019, methodology): “In addition, the deployed recommendation models follow a golden paradigm of embedding 1 and Multi-Layer Perceptron (MLP) [3]: user data are ﬁrst encoded into high-dimensional sparse feature vectors, then embedded into low-dimensional dense vectors, and ﬁnally fed into fully connected layers.”
Explainable Recommendation: A Survey and New Perspectives (2018, background): “Cheng et al. (2019b) attempted to enrich topic models based on multi-”
AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations (2020, methodology): “Figure 1 illustrates how the recommendation performance of a typical DLRS [5] with embedding dimensions 2, 16 and 128 in terms of the mean-squared-error (MSE) and accuracy changes when we vary x .”
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (2018, background): “Software systems employing deep neural networks are now applied widely in industry, powering the vision systems in social networks [47] and self-driving cars [5], providing assistance to radiologists [24], underpinning recommendation engines used by online platforms [9, 12], enabling the bestperform”
IIRC: Incremental Implicitly-Refined Classification (2020, background): “Deep learning algorithms have led to transformational breakthroughs in computer vision [12, 17], natural language processing [19, 50], speech processing [3, 5], reinforcement learning [36, 44], robotics [16, 1], recommender systems [11, 18], etc.”
Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time (2017, background): “Many state-of-the-art web-scale recommendation systems are content-based, often using deep neural networks [6, 8, 11, 29].”