Ask the GRU: Multi-task Learning for Deep Text Recommendations

By Trapit Bansal, David Belanger, Andrew McCallum. 2016

327 citations recorded by the source corpus.

In a variety of application domains the content to be recommended to users is associated with text. This includes research papers, movies with associated plot summaries, news articles, blog posts, etc. Recommendation approaches based on latent factor models can be extended naturally to leverage text by employing an explicit mapping from text to factors. This enables recommendations for new, unseen content, and may generalize better, since the factors for all items are produced by a compactly-parametrized model. Previous work has used topic models or averages of word embeddings for this mapping. In this paper we present a method leveraging deep recurrent neural networks to encode the text sequence into a latent vector, specifically gated recurrent units (GRUs) trained end-to-end on the collaborative filtering task. For the task of scientific paper recommendation, this yields models with significantly higher accuracy. In cold-start scenarios, we beat the previous state-of-the-art, all of which ignore word order. Performance is further improved by multi-task learning, where the text encoder network is trained for a combination of content recommendation and item metadata prediction. This regularizes the collaborative filtering model, ameliorating the problem of sparsity of the observed rating matrix.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Qiang Yang
3. Mohit Bansal
4. Yi Tay
5. Quoc V. Le
6. Dacheng Tao
7. Yoshua Bengio
8. Jianfeng Gao

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Meng Wang
4. Le Wu
5. Kun Zhang
6. Qiang Yang
7. Zhe Zhao
8. Ed H. Chi

In Natural Language Processing

1. Kyunghyun Cho
2. Weinan Zhang
3. Mohit Bansal
4. Yi Tay
5. Jun Wang
6. Yong Yu
7. Arthur Szlam
8. Isabelle Augenstein

Citations in the corpus

• On Sampling Strategies for Neural Network-based Collaborative Filtering (2017, methodology): “Many existing algorithms with graph-based loss functions [1, 22, 29] adopt the “Negative Sampling” strategy, in which k negative samples are drawn whenever a positive example is drawn.”
• Automatic Rule Extraction from Long Short Term Memory Networks (2016, background): “Another approach that has emerged in the literature Alikaniotis et al. (2016) Denil et al. (2015) Bansal et al. (2016) is for each word in the document, looking at the norm of the derivative of the loss function with respect to the embedding parameters for that word.”
• Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts (2018, background): “In [3], a text recommendation task is improved by sharing feature representations and lower level hidden layers.”
• Neural Input Search for Large Scale Recommendation Models (2019, background): “For large scale recommendation problems, great results have also been reported by leveraging advanced representation learning components, such as CNN ([10], [18]), RNN ([1], [8]), etc.”
• Dynamic Attention Deep Model for Article Recommendation by Learning Human Editors' Demonstration (2017, background): “For textual classi€cation and recommendation tasks, deep learning also delivers convincing performance [5, 14, 20, 44, 53].”
• Gumbel-Matrix Routing for Flexible Multi-task Learning (2019, background): “1 INTRODUCTION Multi-task learning (Caruana, 1998; 1993) based on neural networks has attracted lots of research interest in the past years and has been successfully applied to several application domains, such as recommender systems (Bansal et al., 2016) and real-time object detection (Girshick, 20”
• Recommendation system based on deep learning methods: a systematic review and new directions (2019, methodology): “Hidasi et al. (2016b), Bansal et al. (2016), Jannach and Ludewig (2017) and Ludewig and Jannach (2018) exploited RNN architecture to better extract features for the user’s immediate next actions recommendation.”
• A Diagnostic Study of Explainability Techniques for Text Classification (2020, methodology): “As we found different aggregation approaches in related studies (Bansal et al., 2016; DeYoung et al., 2020), we employ the two most common methods – L2 norm and averaging (denoted as μ and `2 in the explainability method names).”
• Small Towers Make Big Differences (2020, background): “…multi-task deep learning has gained popularity through its success in a wide range of applications, including natural language processing (Collobert and Weston, 2008), computer vision (Girshick, 2015; Ren et al., 2015), and online recommendation systems (Bansal et al., 2016; Ma et al., 2018, 2019).”
• Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations (2020, background): “The task of text recommendation in [1] is improved through sharing representations at the bottom.”
• Transfer Meets Hybrid: A Synthetic Approach for Cross-Domain Collaborative Filtering with Text (2019, background): “l for recommendation beyond user-item interaction data. For application domains like recommending research papers and news articles, the unstructured text associated with the item is its text content [1, 51]. Other domains like recommending products, the unstructured text associated with the item is”
• A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, methodology): “based models, researchers also employed various state-ofthe-art content embedding techniques, such as RNNs for item content representation [90].”
• Joint Text Embedding for Personalized Content-based Recommendation (2017, methodology): “…instead of a rating prediction problem, thus pairwise loss functions are adopted; 2) our model provide a more general framework, enabling various text embedding functions, thus subsumes [1] as a special case; 3) our model incorporates unsupervised text embedding from large-scale unlabeled corpora.”
• Deep Learning Based Recommender System (2017, methodology): “r based collaborative recommendation model. Instead of using the original partial observed vectors, it decomposes them by integer ratings. For example, if the ratings score is integer in the range of [1, 2, 3, 4, 5], each r„i”will be divided into •ve partial vectors. Figure 4(c) presents an example ”