A Neural Autoregressive Approach to Collaborative Filtering

By Yin Zheng, Bangsheng Tang, Wenkui Ding, Hanning Zhou. 2016

227 citations recorded by the source corpus.

This paper proposes CF-NADE, a neural autoregressive architecture for collaborative filtering (CF) tasks, which is inspired by the Restricted Boltzmann Machine (RBM) based CF model and the Neural Autoregressive Distribution Estimator (NADE). We first describe the basic CF-NADE model for CF tasks. Then we propose to improve the model by sharing parameters between different ratings. A factored version of CF-NADE is also proposed for better scalability. Furthermore, we take the ordinal nature of the preferences into consideration and propose an ordinal cost to optimize CF-NADE, which shows superior performance. Finally, CF-NADE can be extended to a deep model, with only moderately increased computational complexity. Experimental results show that CF-NADE with a single hidden layer beats all previous state-of-the-art methods on MovieLens 1M, MovieLens 10M, and Netflix datasets, and adding more hidden layers can further improve the performance.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

On Sampling Strategies for Neural Network-based Collaborative Filtering (2017, background): “Hence, it is natural to combine deep learning with traditional collaborative filtering for recommendation tasks, as seen in recent studies [1, 4, 32, 37].”
Variational Autoencoders for Collaborative Filtering (2018, background): “tent factor models can significantly boost recommendation performance. Recently, a growing body of work involves applying neural networks to the collaborative filtering setting with promising results [14, 41, 51, 54]. Here, we extend variational autoencoders (vaes) [24, 37] to collaborative filterin”
Neural Collaborative Filtering (2017, methodology): “[48] presented a neural autoregressive method for CF.”
Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks (2017, background): “Recent works applying deep learning to recommender systems used networks with fully connected or auto-encoder architectures [44, 35, 14].”
GLocal-K: Global and Local Kernels for Recommender Systems (2021, methodology): “(3) CF-NADE[13] replaces the role of the restricted Boltzmann machine (RBM) with the neural auto-regressive distribution estimator (NADE) for rating reconstruction.”
Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources (2017, methodology): “For example, [25, 38] adopted denoising auto-encoders for recommendation, [46] developed a neural autoregressive approach for collaborative ltering, and [19, 20] generalized matrix factorization and factorization machines for neural collaborative ltering.”
Embarrassingly Shallow Autoencoders for Sparse Data (2019, background): “g, [5, 7–9, 13, 21, 25, 26].”
Bridging Collaborative Filtering and Semi-Supervised Learning: A Neural Approach for POI Recommendation (2017, background): “[32] later extends it to accommodate the ordinal nature of ratings and [57] further improves it with the autoregressive method.”
Deep Models of Interactions Across Sets (2018, methodology): “…(ours) 0.863 Factorized EAE (ours) 0.860 I-RBM (Salakhutdinov et al., 2007) 0.854 BiasMF (Koren et al., 2009) 0.845 NNMF (Dziugaite & Roy, 2015) 0.843 LLORMA-Local (Lee et al., 2013) 0.833 GC-MC (Berg et al., 2017) 0.832 I-AUTOREC (Sedhain et al., 2015) 0.831 CF-NADE (Zheng et al., 2016) 0.829”
A review on deep learning for recommender systems: challenges and remedies (2018, methodology): “…results show that the proposed algorithm transcends the state-of-the-art methods including some matrix factorization-based methods and neural network-based methods such as AutoRec (Sedhain et al. 2015) and CF-NADE (Zheng et al. 2016b) onMovieLens and Netflix datasets regarding accuracy.”
Deep Learning Based Recommender System (2017, unknown): “] [36], [37], [121]. [114] RNN [4], [53], [92], [18] [64], [101], [123], [49] [41], [125], [124], [40] [98], [100], [75], [127] [122], [23] [19], [102], [79] [94] DSSM [26], [28], [8], [129] * * NADE [139], [138] * * GAN * [113] * Deep Composite Models [59], [83], [135], [95] [25], [61], [58], [117]”
Joint Text Embedding for Personalized Content-based Recommendation (2017, methodology): “Methods like matrix factorization [14, 24] are wide adopted, and recently some methods based on neural networks are also explored [26, 31, 33].”
Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System (2020, methodology): “Five real-world benchmark datasets are used in our experiments: ML10M is the widely-used [6, 35, 49] dataset Algorithm 1 Inference”
Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information (2017, methodology): “[45] proposed a neural autoregressive distribution estimator for CF (CF-NADE) which builds on Neural Autoregressive Distribution Estimator (NADE) [19].”
NAT: Neural Architecture Transformer for Accurate and Compact Architectures (2019, background): “Deep neural networks (DNNs) [25] have been producing state-of-the-art results in many challenging tasks including image classification [12, 23, 42, 57, 58, 18, 11, 53], face recognition [38, 43, 56], brain signal processing [33, 34], video analysis [50, 49] and many other areas [55, 54, 24, 3, 10, 9”
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search (2020, unknown): “…many challenging tasks including image classification (Krizhevsky et al., 2012; Liu et al., 2020; Zheng et al., 2015; Guo et al., 2020a), semantic segmentation (Shelhamer et al., 2017; Chen et al., 2018; Huang et al., 2019), and many other areas (Zheng et al., 2016b;a; Jiang *Equal contribution .”
UvA-DARE ( Digital Academic Repository ) Graph Convolutional Matrix Completion (2017, unknown): “We shall note at this point that several recent stateof-the-art models for matrix completion [17, 27, 7, 32] can be cast into this framework and understood as a special case of our model.”