A Generic Coordinate Descent Framework for Learning from Implicit Feedback

By Immanuel Bayer, Xiangnan He, Bhargav Kanagal, Steffen Rendle. 2016

233 citations recorded by the source corpus.

In recent years, interest in recommender research has shifted from explicit feedback towards implicit feedback data. A diversity of complex models has been proposed for a wide variety of applications. Despite this, learning from implicit feedback is still computationally challenging. So far, most work relies on stochastic gradient descent (SGD) solvers which are easy to derive, but in practice challenging to apply, especially for tasks with many items. For the simple matrix factorization model, an efficient coordinate descent (CD) solver has been previously proposed. However, efficient CD approaches have not been derived for more complex models. In this paper, we provide a new framework for deriving efficient CD algorithms for complex recommender models. We identify and introduce the property of k-separable models. We show that k-separability is a sufficient property to allow efficient optimization of implicit recommender problems with CD. We illustrate this framework on a variety of state-of-the-art models including factorization machines and Tucker decomposition. To summarize, our work provides the theory and building blocks to derive efficient implicit CD algorithms for complex recommender models.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Tat-seng Chua
3. Xiang Wang
4. Liqiang Nie
5. Fuli Feng
6. Yi Tay
7. Xing Xie
8. Yong Li

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Liqiang Nie
5. Fuli Feng
6. Xing Xie
7. Yi Tay
8. Yong Li

In Management Science and Operations Research

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Fuli Feng
5. Zhe Zhao
6. Ed H. Chi
7. Yi Tay
8. Lichan Hong

Citations in the corpus

• Neural Collaborative Filtering (2017, methodology): “In future, we will study pairwise learners for NCF models and extend NCF to model auxiliary information, such as user reviews [11], knowledge bases [45], and temporal signals [1].”
• NAIS: Neural Attentive Item Similarity Model for Recommendation (2018, background): “Later research efforts on CF have shifted towards learning recommenders from implicit feedback [1], [5], [6], [8], [11].”
• Sampling-bias-corrected neural modeling for large corpus item recommendations (2019, methodology): “In matrix factorization, such a loss is typically trained using alternating least squares [23] or coordinate descent methods [3] by writing the regularization term as a matrix-inner-product of two Gramians, which can be computed efficiently for linear embeddings.”
• DELF: A Dual-Embedding based Deep Latent Factor Model for Recommendation (2018, methodology): “To evaluate the recommendation performance, we employed the widely used leave-one-out evaluation [Rendle et al. , 2009; He et al. , 2017; Bayer et al. , 2017].”
• Self-supervised Graph Learning for Recommendation (2020, background): “However, the observed interactions are extremely sparse [3, 18] compared to the whole interaction space, making it insufficient to learn quality representations.”
• λOpt: Learn to Regularize Recommender Models in Finer Levels (2019, methodology): “We illustrate our approach on matrix factorization, which is representative of a large family of embedding-based recommender models [3].”
• Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems (2020, methodology): “[3, 13, 18] utilize user’s implicit feedback, commonly optimizing BPR loss [30].”
• Outer Product-based Neural Collaborative Filtering (2018, methodology): “Many extensions have been developed for MF from both the modeling perspective [Wang et al., 2015; Yu et al., 2018; Wang et al., 2018a] and learning perspective [Rendle et al., 2009; Bayer et al., 2017; He et al., 2018].”
• EMD2FNN: A strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction (2019, background): “It has brought a plethora 150 of successful applications in industry, and has yielded great promise in a variety of prediction tasks, such as regression, classification and ranking (Oentaryo et al., 2014; Chen et al., 2016; Juan et al., 2016; Bayer et al., 2017).”
• Adversarial Personalized Ranking for Recommendation (2018, methodology): “This is fundamentally different from pointwise methods [2, 17] that optimize each model prediction towards a predefined groundtruth.”
• Sequential Recommender System based on Hierarchical Attention Networks (2018, background): “Therefore, it is not appropriate to directly optimize such one-class score (i.e., 1 or 0) through conventional latent factor model [Bayer et al. , 2017].”
• TEM: Tree-enhanced Embedding Model for Explainable Recommendation (2018, background): “FM [5, 32] is a representative of such predictive models, which is formulated as:”
• Neural Factorization Machines for Sparse Predictive Analytics (2017, background): “Predictive analytics is one of the most important techniques for many information retrieval (IR) and data mining (DM) tasks, ranging from recommendation systems [2, 16], targeted advertising [21], to search ranking [19, 39], visual analysis [35], and event detection [40].”
• Deep Item-based Collaborative Filtering for Top-N Recommendation (2018, methodology): “What is more, the work in Reference [8] extends item similarities to high orders and captures higher-order item relations by integrating itemsets to SLIM.”
• Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking (2017, background): “Since it is too time consuming to rank all items for every user, we randomly sampled 100 items that have no interactions with the target user and ranked the test item with respect to these 100 items. is in concert with many works [2, 6, 7, 21].”