Bayesian Matrix Factorization with Side Information and Dirichlet Process Mixtures

By Ian R. Porteous, Arthur U. Asuncion, Max Welling. 2010

149 citations recorded by the source corpus.

Matrix factorization is a fundamental technique in machine learning that is applicable to collaborative filtering, information retrieval and many other areas. In collaborative filtering and many other tasks, the objective is to fill in missing elements of a sparse data matrix. One of the biggest challenges in this case is filling in a column or row of the matrix with very few observations. In this paper we introduce a Bayesian matrix factorization model that performs regression against side information known about the data in addition to the observations. The side information helps by adding observed entries to the factored matrices. We also introduce a nonparametric mixture model for the prior of the rows and columns of the factored matrices that gives a different regularization for each latent class. Besides providing a richer prior, the posterior distribution of mixture assignments reveals the latent classes. Using Gibbs sampling for inference, we apply our model to the Netflix Prize problem of predicting movie ratings given an incomplete user-movie ratings matrix. Incorporating rating information with gathered metadata information, our Bayesian approach outperforms other matrix factorization techniques even when using fewer dimensions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Recurrent Latent Variable Networks for Session-Based Recommendation (2017, methodology): “To this end, we draw inspiration from recent RS developments which rely on the utilization of Bayesian inference techniques (e.g., [5, 6, 12, 22, 27]).”
Modeling Consumer Preferences and Price Sensitivities from Large-Scale Grocery Shopping Transaction Logs (2017, methodology): “Of particular interest, feature-based matrix factorization techniques have been proposed [3,4,27,28,30] and efficient tools (e.”
A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems (2017, background): “Various models show that additional side information can act as a useful informative prior that can significantly improve results (Porteous, Asuncion, and Welling 2010; Singh and Gordon 2008).”
Seismic sparse-spike deconvolution via Toeplitz-sparse matrix factorization (2016, methodology): “These models have already been successfully applied to numerous problems, e.g., collaborative filtering (Lim and Teh, 2007; Porteous et al., 2010), distinguishing structure from motion (Tomasi and Kanade, 1992; Bregler et al., 2000), and blind source separation (Hyvärinen and Oja, 2000; Cichocki et…”
Speedup Matrix Completion with Side Information: Application to Multi-Label Learning (2013, background): “Although several studies exploit side information for matrix recovery [1, 2, 3, 16, 29, 32, 33], most of them focus on matrix factorization techniques, which usually result in non-convex optimization problems without guarantee of perfectly recovering the target matrix.”
Collaborative Filtering beyond the User-Item Matrix (2014, methodology): “We also mention here another category of algorithms [Porteous et al. 2010; Shan and Banerjee 2010], which can be regarded as being related to those mentioned earlier.”
Improving Content-based and Hybrid Music Recommendation using Deep Learning (2014, methodology): “Although such approaches can potentially generalize to music recommendation, we do not use them due to efficiency issues: (1) they use full Bayesian inference [20, 22, 23] or Monte Carlo simulation [24] and are thus much slower than our algorithm; (2) they have been applied to a dataset with thousan”
Content-based recommendations with Poisson factorization (2014, background): “The q u are independent of topics and offer greater modeling ﬂexibility, but they are less interpretable than the η u in CTPF. Decoupling the factorizations has been proposed by Porteous et al. [16].”
A hierarchical model for ordinal matrix factorization (2011, methodology): “Most other systems require a Gaussian likelihood: Porteous et al. (2010)’s BMFSI also regresses against user and item features, with the addition of Dirichlet process mixture priors on u m and v n .”
Nonparametric bayesian multitask collaborative filtering (2013, methodology): “That paper essentially builds upon a previous work on Bayesian matrix factorization [24], where the user and item factor matrices are considered to be drawn from a Dirichlet process mixture prior [21], instead of simple multivariate Gaussians.”