Tensor-Train Decomposition

By Ivan V. Oseledets. 2011

3,145 citations recorded by the source corpus.

Subfields: Atomic and Molecular Physics, and Optics, Computational Mathematics, Computational Theory and Mathematics, Representation & Self-Supervised Learning, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Computational Mathematics

In Computational Theory and Mathematics

In Representation & Self-Supervised Learning

In Mathematical Optimization

Citations in the corpus

Initialization and Regularization of Factorized Neural Layers (2021, neutral): “(2019) found that Tensor-Train decomposition (Oseledets, 2011) obtained the best memory-accuracy trade-off on WideResNet; we thus compare directly to this approach.”
A literature survey of low‐rank tensor approximation techniques (2013, positive): “In the context of numerical analysis, a decomposition of the form (5) was ﬁrst proposed in [212, 213, 217].”
Recent advances in convolutional neural networks (2015, methodology): “ations to reduced the required computations. Novikov et al . [163] generalize the low-rank ideas, where they treat the weight matrix as multi-dimensional tensor and apply a Tensor-Train decomposition [164] to reduce the number of parameters of the fully-connected layers. Adaptive Fastfood transform ”
Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “In contrast, tensor-wise examples include SVD decomposition [9,64], CPdecomposition [28], Tucker decomposition [22], Tensor-Train decomposition [45],”
Deep Multi-task Representation Learning: A Tensor Factorisation Approach (2016, methodology): “Unlike matrix factorisation, there are multiple deﬁnitions of tensor factorisation, and we use Tucker [Tucker, 1966] and Tensor Train (TT) [Oseledets, 2011] decompositions.”
Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition (2017, methodology): “Tensor Train (TT) decomposition, proposed in Oseledets (2011), allows to efficiently store tensors (multidimensional arrays of data), large matrices, and vectors.”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, neutral): “1Although similar approaches have been known in quantum physics for a long time, their rigorous mathematical analysis is still a work in progress (see (Oseledets, 2011; Orús, 2014) and references therein).”
Low-rank solvers for unsteady Stokes–Brinkman optimal control problem with random data (2016, neutral): “Generally speaking, optimization1 problems constrained by unsteady partial differential equations (PDEs) are a lot more computationally2 challenging because one needs to solve a system of PDEs coupled globally in time and space, and time-3 stepping methods quickly reach their limitations due to the…”
Unsupervised Generative Modeling Using Matrix Product States (2017, background): “The same representation is referred to as tensor train decomposition in the applied math community [28].”
T-Basis: a Compact Representation for Neural Networks (2020, methodology): “One popular structure of TNs for neural network compression is Tensor-Train (TT) (Oseledets, 2011), which is equivalent to MPS. TT decomposes a tensor into a set of third-order tensors, which resembles a linear “train” structure, and has been used for compressing the fully-connected (Novikov et…”
Optimization problems in contracted tensor networks (2011, methodology): “A TT tensor is a TC tensor where one representation rank component is equal to one, so we will perform the conversion with the procedure which is described in [15] as TT-SVD algorithm and in [21] as Vidal decomposition in the MPS context where we choose an approximation accuracy of 10−12.”
Multilevel Toeplitz Matrices Generated by Tensor-Structured Vectors and Convolution with Logarithmic Complexity (2013, background): “QTT ranks of a tensor are defined as the TT ranks of the tensor subject to a proper reshaping.”
A Tensorized Transformer for Language Modeling (2019, methodology): “Tensorizing Neural Networks [24] have combined the idea of reshaping weights of fully-connected layers into high-dimensional tensors and representing them in Tensor Train format [25].”
Trace Norm Regularised Deep Multi-Task Learning (2016, methodology): “matrix to tensor is not unique, just like tensor rank has multiple deﬁnitions. How to deﬁne tensor rank depends on how we assume the tensor is factorised, e.g., Tucker (Tucker, 1966) and Tensor-Train Oseledets (2011) decompositions. We propose three tensor trace norm designs here, which correspond t”
Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation (2012, background): “It is therefore easily accessible and in principle applicable to more delicate types of redundancies as they, for instance, occur in the Tucker format or in the newly developed TT format [19, 20].”
In Teacher We Trust: Learning Compressed Models for Pedestrian Detection (2016, background): “…function L used for training the student is a combination of the soft loss L soft , the cross-entropy loss be-tween the soft outputs of the student and teacher, as well as the hard loss L hard , the standard classiﬁcation cross-entropy loss between the student outputs and the ground truth labels.”
Randomized Matrix Computations (2012, neutral): “eleration of fundamental matrix computations by means of tensor decomposition. More special direction of Tensor Train (TT) decomposition was proposed in [O09] and further developed in [OT09], [OT10], [O11]. It is closely related to the DMRG quantum computations [V03], [W93] and Hierarchical Tucker (”
Long-term Forecasting using Higher Order Tensor RNNs (2017, background): “o smooth tensors (Orus, 2014). The most commonly used tensor networks are linear tensor networks (LTN), also known as tensor-trains in numerical analysis or matrix-product states in quantum physics (Oseledets, 2011). A tensor train model decomposes a P-dimensional tensor Winto a network of sparsely ”
Geometric Structures in Tensor Representations (Final Release) (2013, positive): “It is possible to show that the Tensor Train format [29] (TT for short), introduced originally by Vidal [35], is a particular case of the HT format (see e.g. Chapter 12 in [19]).”