Neural Network Matrix Factorization

By Gintare Karolina Dziugaite, Daniel M. Roy. 2015

188 citations recorded by the source corpus.

Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function---namely, the inner product---acting on the latent feature vectors for the corresponding row and column. Here we consider replacing the inner product by an arbitrary function that we learn from the data at the same time as we learn the latent feature vectors. In particular, we replace the inner product by a multi-layer feed-forward neural network, and learn by alternating between optimizing the network for fixed latent features, and optimizing the latent features for a fixed network. The resulting approach---which we call neural network matrix factorization or NNMF, for short---dominates standard low-rank techniques on a suite of benchmark but is dominated by some recent proposals that take advantage of the graph features. Given the vast range of architectures, activation functions, regularizers, and optimization techniques that could be used within the NNMF framework, it seems likely the true potential of the approach has yet to be reached.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Neural Autoregressive Approach to Collaborative Filtering (2016, background): “Recently, Dziugaite & Roy (2015) proposed Neural Network Matrix Factorization (NNMF), where the inner product between the vectors of users and items in MF is replaced by a feed-forward neural network.”
Neural Collaborative Filtering vs. Matrix Factorization Revisited (2020, methodology): “on to apply a dot product at the output for multiclass classication. 6.2 MLPs at the Output Layer of DNNs NeuMF is very closely related to the previously proposed neural network matrix factorization [11]. Neural network matrix factorization also uses a combination of an MLP plus extra embeddings wit”
Transfer Meets Hybrid: A Synthetic Approach for Cross-Domain Collaborative Filtering with Text (2019, background): “trix. Neural networks are proposed to push the learning of feature vectors towards non-linear representations, including the neural network matrix factorization (NNMF) and multilayer perceptron (MLP) [10, 17]. The basic MLP architecture is extended to regularize the factors of users and items by soc”
Deep Models of Interactions Across Sets (2018, methodology): “…Self-Supervised Model (ours) 0.863 Factorized EAE (ours) 0.860 I-RBM (Salakhutdinov et al., 2007) 0.854 BiasMF (Koren et al., 2009) 0.845 NNMF (Dziugaite & Roy, 2015) 0.843 LLORMA-Local (Lee et al., 2013) 0.833 GC-MC (Berg et al., 2017) 0.832 I-AUTOREC (Sedhain et al., 2015) 0.831 CF-NADE…”
UvA-DARE ( Digital Academic Repository ) Graph Convolutional Matrix Completion (2017, unknown): “We shall note at this point that several recent stateof-the-art models for matrix completion [17, 27, 7, 32] can be cast into this framework and understood as a special case of our model.”
Deep Feature Factorization For Concept Discovery (2018, methodology): “There has been work extending NMF to multiple layers [6], implementing NMF using neural networks [9] and using NMF approximations as input to a neural network [34].”
CoNet: Collaborative Cross Networks for Cross-Domain Recommendation (2018, methodology): “CMF, however, is a shallow model and has the difficulty in learning the complex user-item interaction function [9, 12].”
Deep Learning Based Recommender System (2017, background): “Neural Network Matrix Factorization (NNMF) [37] and Neural Collaborative Filtering (NCF) [53] are two representative works.”