Tensor Ring Decomposition

By Qibin Zhao, Guoxu Zhou, Shengli Xie, Liqing Zhang, Andrzej S. Cichocki. 2016

449 citations recorded by the source corpus.

Tensor networks have in recent years emerged as the powerful tools for solving the large-scale optimization problems. One of the most popular tensor network is tensor train (TT) decomposition that acts as the building blocks for the complicated tensor networks. However, the TT decomposition highly depends on permutations of tensor dimensions, due to its strictly sequential multilinear products over latent cores, which leads to difficulties in finding the optimal TT representation. In this paper, we introduce a fundamental tensor decomposition model to represent a large dimensional tensor by a circular multilinear products over a sequence of low dimensional cores, which can be graphically interpreted as a cyclic interconnection of 3rd-order tensors, and thus termed as tensor ring (TR) decomposition. The key advantage of TR model is the circular dimensional permutation invariance which is gained by employing the trace operation and treating the latent cores equivalently. TR model can be viewed as a linear combination of TT decompositions, thus obtaining the powerful and generalized representation abilities. For optimization of latent cores, we present four different algorithms based on the sequential SVDs, ALS scheme, and block-wise ALS techniques. Furthermore, the mathematical properties of TR model are investigated, which shows that the basic multilinear algebra can be performed efficiently by using TR representaions and the classical tensor decompositions can be conveniently transformed into the TR representation. Finally, the experiments on both synthetic signals and real-world datasets were conducted to evaluate the performance of different algorithms.

Subfields: Computational Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mathematics

1. Yuan Xie
2. Song Han
3. By Lei Deng
4. Luc Van Gool
5. Guoqi Li
6. Dengxin Dai
7. David Bruggemann
8. Suman Saha

Citations in the corpus

• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “In contrast, tensor-wise examples include SVD decomposition [9,64], CP-decomposition [28], Tucker decomposition [22], Tensor-Train decomposition [45], Tensor-Ring decomposition [68], T-Basis [44], etc.”
• Hyperspectral Image Denoising via Tensor Low-Rank Prior and Unsupervised Deep Spatial–Spectral Prior (2022, background): “Definition 2 (Multilinear Product [49]): The multilinear product of G(n) ∈ Rrn−1×In×rn and G(n+1) ∈ Rrn×In+1×rn+1 is denoted G(n,n+1) ∈ Rrn−1×In In+1×rn+1 , and its ik + In+1( jk − 1)th lateral slice satisfies”
• Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey (2020, background): “Although TC seems to have a greater potential to obtain better quality than TT [118], [137], more research studies are needed to evidence this point in neural networks. h) TT in neural networks: In tensor network decomposition, the TT format is widely applied in neural networks.”
• T-Basis: a Compact Representation for Neural Networks (2020, methodology): “Another structure of TNs, with arguably better generalization capabilities (Wang et al., 2017), is Tensor-Ring (TR) (Perez-Garcia et al., 2007; Khoromskij, 2011; Espig et al., 2011; Zhao et al., 2016).”