Tensor Factorization via Matrix Factorization

By Volodymyr Kuleshov, Arun Tejasvi Chaganty, Percy Liang. 2015

86 citations recorded by the source corpus.

Tensor factorization arises in many machine learning applications, such knowledge base modeling and parameter estimation in latent variable models. However, numerical methods for tensor factorization have not reached the level of maturity of matrix factorization methods. In this paper, we propose a new method for CP tensor factorization that uses random projections to reduce the problem to simultaneous matrix diagonalization. Our method is conceptually simple and also applies to non-orthogonal and asymmetric tensors of arbitrary order. We prove that a small number random projections essentially preserves the spectral information in the tensor, allowing us to remove the dependence on the eigengap that plagued earlier tensor-to-matrix reductions. Experimentally, our method outperforms existing tensor factorization methods on both simulated data and two real datasets.

Subfields: Computational Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mathematics

Citations in the corpus

Beyond CCA: Moment Matching for Multi-View Models (2016, neutral): “Importantly, the joint diagonalization problem we deal with in this paper is conceptually different from the one considered, e.g., by Kuleshov et al. (2015) (and references therein) and, therefore, the respective algorithms are not applicable here.”
Rethinking LDA: Moment Matching for Discrete ICA (2015, positive): “While already working on this manuscript, the JD algorithm was also independently analyzed by [27] in the context of tensor factorization for general latent variable models.”
Recovery Guarantees for One-hidden-layer Neural Networks (2017, methodology): “After that, the non-orthogonal tensor decomposition (KCL)(Kuleshov et al., 2015) on (cid:98) R 3 outputs (cid:98) u i which estimates s i V (cid:62) w ∗ i for i ∈ [ k ] with unknown sign s i ∈ {− 1 , 1 } .”
Dynamical Variational Autoencoders: A Comprehensive Review (2020, methodology): “FVAE: The authors of (Deng et al., 2017) propose a model called Factorized VAE (FVAE) that combines the VAE with tensor factorization (Kuleshov et al., 2015; Huang et al., 2016), this latter being applied on the latent vector z.”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, unknown): “Although a uniform random sampling may experiences problem for tensors with spiky elements, this approach often outperforms the standard CP-ALS decomposition algorithms [161].”
Online and Differentially-Private Tensor Decomposition (2016, background): “Acknowledgement A. Anandkumar is supported in part by Microsoft Faculty Fellowship, NSF Career award CCF-1254106, ONR Award N00014- 14-1-0665, ARO YIP Award W911NF-13-1-0084 and AFOSR YIP FA9550-15-1-0221.”