Latent Cross: Making Use of Context in Recurrent Recommender Systems

By Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, Ed H. Chi. Web Search and Data Mining, 2018

371 citations recorded by the source corpus.

The success of recommender systems often depends on their ability to understand and make use of the context of the recommendation request. Significant research has focused on how time, location, interfaces, and a plethora of other contextual features affect recommendations. However, in using deep neural networks for recommender systems, researchers often ignore these contexts or incorporate them as ordinary features in the model. In this paper, we study how to effectively treat contextual data in neural recommender systems. We begin with an empirical analysis of the conventional approach to context as features in feed-forward recommenders and demonstrate that this approach is inefficient in capturing common feature crosses. We apply this insight to design a state-of-the-art RNN recommender system. We first describe our RNN-based recommender system in use at YouTube. Next, we offer "Latent Cross," an easy-to-use technique to incorporate contextual data in the RNN by embedding the context feature first and then performing an element-wise product of the context embedding with model's hidden states. We demonstrate the improvement in performance by using this Latent Cross technique in multiple experimental settings.

Subfields: Artificial Intelligence, Information Systems, Signal Processing

View Paper: https://www.semanticscholar.org/paper/629781b0d1cb7d7589b45dc4937ae59ad591ce72

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Signal Processing

Citations in the corpus

Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks (2019, methodology): “nd variants (LSTMs and GRUs) to build recommender systems. RRN [45] uses RNNs to generate dynamic user and item embeddings from rating networks. Recent methods, such as Time-LSTM [52] and LatentCross [8] learn how to incorporate features into the embeddings. However, most of these methods suffer fro”
Self-Attentive Sequential Recommendation (2018, methodology): “Other than MC-based methods, another line of work adopts RNNs to model user sequences [2], [23]–[25].”
Fairness in Recommendation Ranking through Pairwise Comparisons (2019, background): “Two that relate to the challenges in fairness are the temporal dynamics [33, 48, 26, 9] and biased training data [29, 15, 3].”
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks (2018, background): “er feature interactions. However, such kinds of methods suffer from two limitations. First, fully-connected neural networks have been shown inefficient in learning multiplicative feature interactions [4]. Second, since these models learn the feature interactions in an implicit way, they lack good ex”
Time-Dependent Representation for Neural Event Sequence Prediction (2017, methodology): “ect concatenation between the inter-event interval and the respective event embedding as the input to the recurrent layer where the representation of the temporal information is limited. Recent work (Beutel et al., 2018) employed "latent crossing" of event and time interval based on pairwi”
A Survey on Session-based Recommender Systems (2019, background): “There are also other variants that incorporate (1) variational inference into RNN to handle the uncertainty in sparse session data and simultaneously enhance the model’s scalability on large realworld datasets for recommendations [11, 19]; (2) side information like item features and contextual facto”
AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction (2020, positive): “R. The core of these recommender systems is to extract significant low-order and high-order feature interactions. Explicit feature interactions can significantly improve the performance of CTR models [2, 7, 9, 19]. Early collaborative filtering recommendation algorithms, such as matrix factorization”
Neural Collaborative Filtering vs. Matrix Factorization Revisited (2020, background): “are crucial for advancing the state of the art of deep learning, although in theory they can all be approximated by MLPs. The ineciency of MLPs to capture dot and tensor products has been studied by [5] in the context of recommender systems. Here the authors examine how to add context to recurrent n”
Revisiting Deep Learning Models for Tabular Data (2021, methodology): “Inspired by this motivation, some works (Beutel et al., 2018; Wang et al., 2017, 2020) have proposed different ways to incorporate feature products into MLP.”
Outer Product-based Neural Collaborative Filtering (2018, background): “Moreover, it has been recently shown that, modeling the interaction of feature embeddings explicitly is particularly useful for a deep learning model to generalize well on sparse data, whereas using concatenation is sub-optimal [He and Chua, 2017; Beutel et al., 2018].”
Tabular Data: Deep Learning is Not All You Need (2021, methodology): “interactions, which considered different ways to incorporate feature products into the MLP model[Beutel et al., 2018]; and (3) 1D-CNN, which utilizes the advantages of convolutions in tabular data [Baosenguo, 2021].”
Time Interval Aware Self-Attention for Sequential Recommendation (2020, methodology): “RNNs (Recurrent Neural Network) based models like [3, 12, 13, 18] use RNNs to model entire user sequences.”
A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, background): “Based on RRNs, rich context factors were considered, such as the social influence [153], [207], item metadata [208], [250] and multimedia data fusion [251].”
Adversarial Personalized Ranking for Recommendation (2018, methodology): “In addition to the feed-forward NCF framework, recurrent neural networks have also been developed to handle the temporal signal in session-aware recommendation [3, 22].”
Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation (2019, methodology): “The element-wise multiplication, which has been demonstrated to be efficient [2] for feature interaction modeling, is leveraged to combine item embedding qj with its adaptive visual embedding Ii j .”
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications (2018, background): “Future Trends: • Model Exploration: recent studies explore explicitly incorporating time into the event-probability models [7, 73].”