Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

By Andrzej S. Cichocki, Namgil Lee, Ivan V. Oseledets, Anh Huy Phan, Qibin Zhao, Danilo P. Mandic. Found. Trends Mach. Learn., 2016

600 citations recorded by the source corpus.

Modern applications in engineering and data science are increasinglybased on multidimensional data of exceedingly high volume, variety,and structural richness. However, standard machine learning algorithmstypically scale exponentially with data volume and complexityof cross-modal couplings - the so called curse of dimensionality -which is prohibitive to the analysis of large-scale, multi-modal andmulti-relational datasets. Given that such data are often efficientlyrepresented as multiway arrays or tensors, it is therefore timely andvaluable for the multidisciplinary machine learning and data analyticcommunities to review low-rank tensor decompositions and tensor networksas emerging tools for dimensionality reduction and large scaleoptimization problems. Our particular emphasis is on elucidating that,by virtue of the underlying low-rank approximations, tensor networkshave the ability to alleviate the curse of dimensionality in a numberof applied areas. In Part 1 of this monograph we provide innovativesolutions to low-rank tensor network decompositions and easy to interpretgraphical representations of the mathematical operations ontensor networks. Such a conceptual insight allows for seamless migrationof ideas from the flat-view matrices to tensor network operationsand vice versa, and provides a platform for further developments, practicalapplications, and non-Euclidean extensions. It also permits theintroduction of various tensor network operations without an explicitnotion of mathematical expressions, which may be beneficial for manyresearch communities that do not directly rely on multilinear algebra.Our focus is on the Tucker and tensor train TT decompositions andtheir extensions, and on demonstrating the ability of tensor networksto provide linearly or even super-linearly e.g., logarithmically scalablesolutions, as illustrated in detail in Part 2 of this monograph.

Subfields: Computational Mathematics, Computational Theory and Mathematics, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/0033fabb9f9dad4bdaa81470593601d9506d2812

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mathematics

In Computational Theory and Mathematics

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives (2017, methodology): “For more detail regrading HT networks and their generalizations to TTNS, see Part 1 of our monograph [Cichocki et al., 2016].”
The ITensor Software Library for Tensor Network Calculations (2020, unknown): “blockdims = ([2,2],[2,3]) nzblocks = [(1,2),(2,1)] A = randomBlockSparseTensor(nzblocks,blockdims)”
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer (2020, background): “Matrix factorizations, including low-rank factorization via the SVD and various forms of tensor factorization, have been extensively studied in theory and application [8, 9, 27, 46].”
Geometric Methods on Low-Rank Matrix and Tensor Manifolds (2020, background): “For further reading we would like to refer to the several overview articles taking different perspectives on low-rank optimization, see [61, 37, 16, 7, 15, 17, 100], and the monographs [39, 51, 53].”
Unsupervised Generative Modeling Using Matrix Product States (2017, background): “[23] Andrzej Cichocki, Namgil Lee, Ivan Oseledets, Anh-Huy Phan, Qibin Zhao, Danilo P Mandic, et al.”
Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution (2019, background): “Mode-n canonical matrization [40] of tensor M is defined as”