Fast Matrix Factorization for Online Recommendation with Implicit Feedback

By Xiangnan He, Hanwang Zhang, Min-Yen Kan, Tat-seng Chua. 2016

1,073 citations recorded by the source corpus.

This paper contributes improvements on both the effectiveness and efficiency of Matrix Factorization (MF) methods for implicit feedback. We highlight two critical issues of existing works. First, due to the large space of unobserved feedback, most existing works resort to assign a uniform weight to the missing data to reduce computational complexity. However, such a uniform assumption is invalid in real-world settings. Second, most methods are also designed in an offline setting and fail to keep up with the dynamic nature of online data. We address the above two issues in learning MF models from implicit feedback. We first propose to weight the missing data based on item popularity, which is more effective and flexible than the uniform-weight assumption. However, such a non-uniform weighting poses efficiency challenge in learning the model. To address this, we specifically design a new learning algorithm based on the element-wise Alternating Least Squares (eALS) technique, for efficiently optimizing a MF model with variably-weighted missing data. We exploit this efficiency to then seamlessly devise an incremental update strategy that instantly refreshes a MF model given new feedback. Through comprehensive experiments on two public datasets in both offline and online protocols, we show that our implemented, open-source (https://github.com/hexiangnan/sigir16-eals) eALS consistently outperforms state-of-the-art implicit MF methods.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Neural Collaborative Filtering (2017, methodology): “Among the various collaborative ﬁltering techniques, matrix factorization (MF) [14, 21] is the most popular one, which projects users and items into a shared latent space, using a vector of latent features to represent a user or an item.”
Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “Training Heuristic [69], [70], [71], [72], [73], [74], [75], [76] Sampling [30], [43], [74], [77], [78], [79] Exposure-based model [7], [8], [30], [31], [80] Others [5], [6], [81], [82], [83], [84], [85] Position Bias Click models [86], [87], [88], [89], [90], [91], [92] Propensity Score [5], [93], ”
Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems (2020, methodology): “urchasing. The representative methods are Matrix Factorization (MF) [20] and Factorization Machine(FM)[29].NeuralFM[16]andDeepFM[15]haveimproved FM’s representation ability with deep neural networks. [3, 13, 18] utilize user’s implicit feedback, commonly optimizing BPR loss [30]. [7, 8] exploits use”
A Context-Aware User-Item Representation Learning for Item Recommendation (2017, positive): “ACF incorporates the attentive modules into classic CF models with implicit feedback [26, 27].”
Hierarchical Gating Networks for Sequential Recommendation (2019, background): “at all missing data as negative samples [15], or sample negative instances from missing data to learn the pair-wise user preference between positive and negative samples [29]. Recently, several works [11, 22] are proposed to weigh the missing data. In [14, 36], metric learning is applied to compute ”
Outer Product-based Neural Collaborative Filtering (2018, methodology): “By minimizing the BPR loss, we tailor the ONCF framework for correctly predicting the relative orders between interactions, rather than their absolute scores as optimized in pointwise loss [He et al., 2017; 2016b].”
Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking (2017, background): “Since CML and NeuMF have surpassed many other baselines such as WMF [9], eALS [7] and Factorization Machines [20], we do not further report them.”
Unifying Virtual and Physical Worlds (2017, background): “the other is normalized mean squared error [Zhang et al. 2016b; He et al. 2016]:”
Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention (2017, methodology): “As implicit feedback lacks substantial evidence on which items user dislike ( i.e. , negative feedback), existing CF methods [20, 21, 30] with implicit feedback generally focus on how to tap the missing user-item interactions into preference modeling.”
Discrete Collaborative Filtering (2016, methodology): “As moving forward, we are going to apply DCF to various CF applications [27, 13] as well as more generic feature-based factorization approaches [23].”
DELF: A Dual-Embedding based Deep Latent Factor Model for Recommendation (2018, background): “Recent works [ Pan et al. , 2008; Pan and Scholz, 2009; He et al. , 2014; 2016; Volkovs and Yu, 2015] has proposed non-uniform weighting strategies and sampling schemes to carefully select negative feedback from unobserved interactions.”
Video Question Answering via Attribute-Augmented Attention Network Learning (2017, background): “As an emerging kind of recommender system, visual question answering is an important problem for VIR sites, which automatically returns the relevant answer from thereferenced visualcontents according tousers’ postedques-tion [1, 5–7].”
Neural Factorization Machines for Sparse Predictive Analytics (2017, background): “One main power of FM stems from its generality — in contrast to matrix factorization (MF) that models the relation of two entities only [17], FM is a general predictor working with any real valued feature vector for supervised learning.”
TEM: Tree-enhanced Embedding Model for Explainable Recommendation (2018, methodology): “In the literature, embedding-based methods such as matrix factorization [23, 26, 34] is the most popular CF approach, owing to the strong power of embeddings in generalizing from sparse user-item relations.”
Adversarial Personalized Ranking for Recommendation (2018, methodology): “Among the various CF meth-ods, matrix factorization (MF) [18], a special type of latent factor models, is a basic yet most effective recommender model.”
Context-aware Image Tweet Modelling and Recommendation (2016, background): “Keywords image tweets; Twitter; context; recommendation; image semantics; microblog”
NAIS: Neural Attentive Item Similarity Model for Recommendation (2018, methodology): “pling negative feedback frommissing data [36] or treating all missing data as negative feedback [6].”
Deep Matrix Factorization Models for Recommender Systems (2017, methodology): “Evaluation for Recommendation To evaluate the performance of item recommendation, we adopted the leave-one-out evaluation, which has been widely used in the literatures [He et al., 2016; Kingma and Ba, 2014; He et al., 2017].”
Aspect-Aware Latent Factor Model: Rating Prediction with Ratings and Reviews (2018, background): “As we know, in MF, more latent factors will lead to better performance unless overfitting occurs [20, 21]; while the optimal number of latent topics in topic models (e.”