Neural Collaborative Filtering vs. Matrix Factorization Revisited

By Steffen Rendle, Walid Krichene, Li Zhang, John R. Anderson. ACM Conference on Recommender Systems, 2020

500 citations recorded by the source corpus.

Embedding based models have been the state of the art in collaborative filtering for over a decade. Traditionally, the dot product or higher order equivalents have been used to combine two or more embeddings, e.g., most notably in matrix factorization. In recent years, it was suggested to replace the dot product with a learned similarity e.g. using a multilayer perceptron (MLP). This approach is often referred to as neural collaborative filtering (NCF). In this work, we revisit the experiments of the NCF paper that popularized learned similarities using MLPs. First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities. Second, while a MLP can in theory approximate any function, we show that it is non-trivial to learn a dot product with an MLP. Finally, we discuss practical issues that arise when applying MLP based similarities and show that MLPs are too costly to use for item recommendation in production environments while dot products allow to apply very efficient retrieval algorithms. We conclude that MLPs should be used with care as embedding combiner and that dot products might be a better default choice.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/ddd72d2dd303a869ee376c768c21916787f673db

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, result): “While the neural recommendation models have dominated in the recommendation field and claimed substantial improvements over previous models, recent efforts raise questions about their reproducibility and published claims [270], [271], [272], [273], [274].”
One Person, One Model, One World: Learning Continual User Representation without Forgetting (2020, background): “Among them, deep structured semantic models (DSSM) [14], neural networkbased collaborative filtering (NCF) [30], deep or neural factorization machines (DeepFM/NFM) [7, 10] are some of the representative works based on supervised representation learning.”
Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited (2023, methodology): “, two-tower based DSSM [26, 53] and session-based SASRec [27]) equipped with four powerful ME evaluated on three large-scale”
Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022, background): “Though simple, they have been shown to be more effective than many existing neural models on rating prediction and direct recommendation tasks [8, 9, 46].”
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems (2020, methodology): “In light of recent concerns about poor reproducibility of published results [8, 33, 38], we conducted a fair and comprehensive experimental study with extensive hyper-parameter search to properly tune all the baselines and proposed approaches.”
Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation (2020, methodology): “For a fair comparison, we slightly change NeuFM and DeepFM by treating interacted items in S as features and target items as softmax labels, which has no negative effect on the performance [29].”
Knowledge Graph Contrastive Learning for Recommendation (2022, methodology): “Among various techniques, Collaborative Filtering (CF) frameworks become effective solutions to predict users’ preferences, based on the rationale that users with similar interaction behaviors may share similar interests for items [12, 18, 26].”
The Benchmark Lottery (2021, result): “However, even these datasets are tweaked differently in various publications, leading sometimes to contradictory results [Rendle et al., 2020, Zheng et al., 2019].”