Generalized Low Rank Models

By Madeleine Udell, Corinne Horn, Reza Bosagh Zadeh, Stephen P. Boyd. 2014

386 citations recorded by the source corpus.

Principal components analysis (PCA) is a well-known technique for approximating a tabular data set by a low rank matrix. Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types. This framework encompasses many well-known techniques in data analysis, such as nonnegative matrix factorization, matrix completion, sparse and robust PCA, k-means, k-SVD, and maximum margin matrix factorization. The method handles heterogeneous data sets, and leads to coherent schemes for compressing, denoising, and imputing missing entries across all data types simultaneously. It also admits a number of interesting interpretations of the low rank factors, which allow clustering of examples or of features. We propose several parallel algorithms for fitting generalized low rank models, and describe implementations and numerical results.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Principal Components and Regularized Estimation of Factor Models (2017, background): “Some recent papers on SVT are Udell et al. (2016), mation generally leads to estimates that are less variable, but at the cost of bias.”
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer (2020, background): “Matrix factorizations, including low-rank factorization via the SVD and various forms of tensor factorization, have been extensively studied in theory and application [8, 9, 27, 46].”
Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implications (2017, methodology): “lgorithm (e.g. Parikh and Boyd, 2014) for solving (18) and (19) in each iteration, where the 18 projections have closed-form solutions. Similar algorithms have been considered in other works, such as Udell et al. (2016), Zhu et al. (2016) and Bi et al. (2017), for solving optimization problems with ”
Implicit Deep Learning (2019, methodology): “n this case, the sub-problem of updating Ais not convex, but may be addressed by alternating over the factors L;R, similar to what is done in power iteration or generalized low-rank modelling schemes [15]. 7Numerical experiments 7.1Synthetic feedforward neural network Our purpose in this experiment ”