Variational Autoencoders for Collaborative Filtering
By Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, Tony Jebara. The Web Conference, 2018
1,520 citations recorded by the source corpus.
We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research.We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm has information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.
Subfields: Computer Vision and Pattern Recognition, Information Systems, Probabilistic & Bayesian Methods, Recommender Systems, Signal Processing
View Paper: https://arxiv.org/abs/1802.05814
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Xiangnan He
- Tat-seng Chua
- Dacheng Tao
- Xiang Wang
- Luc Van Gool
- Meng Wang
- Trevor Darrell
- Xiaogang Wang
In Information Systems
In Probabilistic & Bayesian Methods
- Brooks Paige
- Jan-Willem van de Meent
- Adji Bousso Dieng
- Hang Li
- Jun Xu
- David M. Blei
- Alexander M. Rush
- Yoon Kim
In Recommender Systems
In Signal Processing
Citations in the corpus
- NSML: Meet the MLaaS platform with a real-world case study (2018, background): “(Yamada et al., 2018; Xiong et al., 2018; Liang et al., 2018; Yu et al., 2018; Min et al., 2017) As the applications of the deep learning grow, the complexity of the tasks for designing and tuning the neural network architectures increases as well as the size of models.”
- Structured Disentangled Representations (2018, methodology): “In settings where we are not primarily interested in inducing disentangled representations, the β -VAE objective has also been used with β 1 in order to improve the quality of reconstructions [Alemi et al., 2016, Engel et al., 2017, Liang et al., 2018].”
- Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation (2021, background): “• Mult-VAE [16] is a variational autoencoder-based recommendation model.”
- On the Equivalence between Positional Node Embeddings and Structural Graph Representations (2019, background): “, 2014a), and some types of neural networks (You et al., 2019; Liang et al., 2018; Tang et al., 2019; Grover et al., 2019).”
- RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms (2020, background): “MultiDAE [37] WWW 2018 MovieLens, Million Song, Netflix”
- Denoising Implicit Feedback for Recommendation (2020, methodology): “To build more robust recommender systems, some auto-encoder based models [20, 30, 33] introduce the denoising techniques.”
- Diffusion Recommender Model (2023, methodology): “Generative models such as Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) have been broadly utilized for personalized recommendation [19, 37, 50].”
- Controllable Multi-Interest Framework for Recommendation (2020, methodology): “We evaluate the performance of all methods under strong generalization [34, 37, 38]: We split all users into training/validation/test sets by the proportion of 8:1:1.”
- Neural Graph Collaborative Filtering (2019, background): “Towards this end, recent efforts [11, 17, 18, 27, 40] focus on exploiting deep learning techniques to enhance the interaction function, so as to capture the nonlinear feature interactions between users and items.”
- LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation (2020, methodology): “As such, collaborative filtering (CF), which focuses on exploiting the past user-item interactions to achieve the prediction, remains to be a fundamental task towards effective personalized recommendation [10, 19, 28, 39].”
- Learning Disentangled Representations for Recommendation (2019, methodology): “Despite their notable success, the existing user behavior-based representation learning methods, such as the recent deep approaches [49, 32, 31, 52, 11, 18], generally neglect the complex interaction among the latent factors behind the users’ decision making processes.”
- Deep Learning for Matching in Search and Recommendation (2018, methodology): “like neural network-based methods (He et al., 2017c; Liang et al., 2018) and graph-based methods (Wang et al.”
- SVP-CF: Selection via Proxy for Collaborative Filtering Data (2021, methodology): “This method is representative of commonly used pre-processing strategies (see e.g. (Liang et al., 2018; He et al., 2017)) to make data suitable for parameter-heavy algorithms.”
- Hierarchical Disentangled Representations (2018, methodology): “In settings where we are not primarily interested in inducing disentangled representations, β-VAE objective has also been used with β < 1 in order to increase the quality of reconstructions (see, e.g., [Alemi et al., 2016; Engel et al., 2017; Liang et al., 2018]).”
- Distilling Structured Knowledge into Embeddings for Explainable and Accurate Recommendation (2019, methodology): “We compare our proposed approach with the collaborative recommendation methods (ItemPop based on item popularity, BPR [29], WMF [14, 24] and VAE [18]) and the hybrid models that exploit both content and collaborative signals for recommendation (LibFM [28], heterogeneous graph embeddingbased model HE”
- Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation (2021, methodology): “• Mult-VAE [14]: Mult-VAE uses a multinomial likelihood for variational autoencoder to improve the performance.”
- Disentangled Graph Collaborative Filtering (2020, background): “Another similar line applies autoencoders on interaction histories to estimate the generative process of user behaviors, such as Mult-VAE [23], AutoRec [31], and CDAE [44].”