Tensor decompositions, alternating least squares and other tales

By Pierre Comon, Xavier Luciani, A. D. Almeida. 2009

519 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mathematics

Citations in the corpus

A literature survey of low‐rank tensor approximation techniques (2013, neutral): “…robustness of gradient-based and Newton-like methods [2,72,74,140,223,224], modifying and improving ALS (alternating least squares) [41, 57, 58, 90, 225], studying the convergence of ALS [51, 194, 253] and reducing the cost of the unfolding operations required during the approximation [222,269].”
Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition (2015, methodology): “However, in practice the most popular approaches like Alternating Least Squares (Comon et al., 2009) or FastICA (Hyvarinen, 1999) try to use a single optimization problem to find all the components.”
Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis (2017, background): “TCA can nevertheless 650 be difficult to optimize if latent factors are approximately linearly dependent [101].”
Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression (2018, methodology): “When our TNNs are used for compression, we project an existing neural network to the class of TNNs with fewer parameters, and in Section 5, we demonstrate this projection naturally corresponds novel compact architecture.”
New ALS Methods With Extrapolating Search Directions and Optimal Step Size for Complex-Valued Tensor Decompositions (2011, methodology): “This method is called “enhanced line search” (ELS) [6], [29], [30].”
Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis (2014, background): “For instance, a low multilinear rank approximation achieved through Tucker decomposition may yield a higher Signal-to-Noise Ratio (SNR) than the SNR in the original raw data tensor, making Tucker decomposition a very natural tool for compression and signal enhancement [7], [8], [24].”
How hard is the tensor rank (2016, background): “We note that the rank decompositions have proved to be useful for studies in statistics [34], signal processing [26], complexity of computation [39], psychometrics [8], linguistics [38], chemometrics [10], and we refer the reader to the monograph [24] for a more complete survey of applications.”
Tensor Factorization via Matrix Factorization (2015, unknown): “based methods [22] are simple, popular, and apply to the non-orthogonal setting, but are known to easily get stuck in local optima [23].”
Learning Two-layer Neural Networks with Symmetric Inputs (2018, methodology): “In practice we find it is more robust to draw 10k random samples from the subspace spanned by the last k right-singular vectors of T and compute the CP decomposition of all the samples (reshaped as matrices and stacked together as a tensor) via alternating least squares (Comon et al., 2009).”
Distributed large-scale tensor decomposition (2014, methodology): “In order to compute CPD, several algorithms have been proposed in the literature, which can be classiﬁed into three main categories: alternating algorithms, derivative based algorithms, and non-iterative algorithms (see e.g. [3, 4, 5]).”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, unknown): “5 Scope and Objectives Review and tutorial papers [51,59,69,153,177,193,236] and books [53,112,160, 239] dealing with TDs and TNs already exist, however, they typically focus on standard models, with no explicit links to very large-scale data processing topics or connections to a wide class of optim”
Accelerated Canonical Polyadic Decomposition Using Mode Reduction (2012, unknown): “We used the technique adopted in [20] to generate collinear components in A (1) and A (2) , i.e., a ( n ) j = v n,j for j = 1 ; and a , where the entries of v n,j were drawn from independent standard normal distributions.”
Tensors : A brief introduction (2014, unknown): “In practice, a tensor A is often assimilated to its array representation A [46], [16], [21], which is generally not so much confusing.”
Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation (2012, unknown): “But any widely used algorithm should desirably also be backed by a local convergence theory.”
CANDECOMP/PARAFAC Decomposition of High-Order Tensors Through Tensor Reshaping (2012, methodology): “Details on numerical complexity of ALS, ELS and Levenberg-Marquardt (LM) algorithm using QR factorization can also be found in [21].”