A New Approach to Collaborative Filtering: Operator Estimation with Spectral Regularization

By Francis R. Bach, Jacob D. Abernethy, Jean-Philippe Vert, Theodoros Evgeniou. 2008

260 citations recorded by the source corpus.

We present a general approach for collaborative filtering (CF) using spectral regularization to learn linear operators mapping a set of "users" to a set of possibly desired "objects". In particular, several recent low-rank type matrix-completion methods for CF are shown to be special cases of our proposed framework. Unlike existing regularization-based CF, our approach can be used to incorporate additional information such as attributes of the users/objects---a feature currently lacking in existing regularization-based CF approaches---using popular and well-known kernel methods. We provide novel representer theorems that we use to develop new estimation methods. We then provide learning algorithms based on low-rank decompositions and test them on a standard CF data set. The experiments indicate the advantages of generalizing the existing regularization-based CF methods to incorporate related information about users and objects. Finally, we show that certain multi-task learning methods can be also seen as special cases of our proposed approach.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Optimization with Sparsity-Inducing Penalties (2011, background): “Using the trace norm as a convex surrogate for rank has many applications in control theory [48], matrix completion [1, 130], multi-task learning [109], or multi-label classification [4], where low-rank priors are adapted.”
Multiple domain user personalization (2011, background): “To prevent information overload, many collaborative filtering and recommendation systems [1, 19, 8, 3, 2] have been developed to filter the information stream and display only information the user may be interested in.”
Active Multi-task Learning via Bandits (2015, methodology): “If the observed output is y then the loss is l(h(x), y), where l is a fixed loss function on R and is assumed that l ∈ [0, 1].”
Semantic Matching in Search (2014, methodology): “P (w|d) = αPLM (w|d) + (1− α)PTM (w|d), where α ∈ [0, 1] is a coefficient and PTM (w|d) is the probability estimated with the topic model.”
Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity (2010, methodology): “The resulting estimation therefore implements a collaborative filtering which incorporates the information from all the signals in the same cluster [1].”
Generalized Low Rank Models (2014, background): “We generate a low rank matrix B = XY œ [0, 1]m◊n with X œ”
Approximation accuracy, gradient methods, and error bound for structured convex optimization (2010, methodology): “This problem has applications to dimension reduction in multivariate linear regression [75] and multi-task learning [1, 3, 106].”
Speedup Matrix Completion with Side Information: Application to Multi-Label Learning (2013, background): “Although several studies exploit side information for matrix recovery [1, 2, 3, 16, 29, 32, 33], most of them focus on matrix factorization techniques, which usually result in non-convex optimization problems without guarantee of perfectly recovering the target matrix.”
Transfer learning in heterogeneous collaborative filtering domains (2013, methodology): “Data set Form Sparsity Moviepilot (subset) target (training) [1,5] ∪ {?} < 1% target (test) [1,5] ∪ {?} 11.”
Revisiting Stein's paradox: multi-task averaging (2014, background): “In this paper we do just that: apply this regularizer to mean estimation, show that this special case enables new and useful analytic results, and demonstrate its performance with simulated and real data.”
Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization (2012, background): “For example, it is shown in [1, 2, 36, 38] that certain multi-task learning problems with least squares loss functions can be exactly captured by (1.”
Low-Rank Matrix Recovery via Efficient Schatten p-Norm Minimization (2012, background): “…machine learning applications such as recommender system and collaborative prediction (Srebro, Rennie, and Jaakkola 2004; Rennie and Srebro 2005; Abernethy et al. 2009), multitask learning (Abernethy et al. 2006; Pong et al. 2010; Argyriou, Evgeniou, and Pontil 2008), image/video completion…”
General Functional Matrix Factorization Using Gradient Boosting (2013, background): “From the perspective of infinite feature space, one related general framework is (Abernethy et al., 2009), which implicitly defines features using kernel functions.”
Multi-Task Learning for Spatio-Temporal Event Forecasting (2015, background): “Suppose we have a predefined subset of keywords of size d in V that are relevant to the event of interest for forecasting, and denote A as the corresponding incidence matrix, A ∈ [0, 1]d×p.”