The geometry of algorithms using hierarchical tensors

By André Uschmajew, Bart Vandereycken. 2013

139 citations recorded by the source corpus.

In this paper, the differential geometry of the novel hierarchical Tucker format for tensors is derived. The set HT,k of tensors with fixed tree T and hierarchical rank k is shown to be a smooth quotient manifold, namely the set of orbits of a Lie group action corresponding to the non-unique basis representation of these hierarchical tensors. Explicit characterizations of the quotient manifold, its tangent space and the tangent space of HT,k are derived, suitable for high-dimensional problems. The usefulness of a complete geometric description is demonstrated by two typical applications. First, new convergence results for the nonlinear Gauss-Seidel method on HT, k are given. Notably and in contrast to earlier works on this subject, the task of minimizing the Rayleigh quotient is also addressed. Second, evolution equations for dynamic tensor approximation are formulated in terms of an explicit projection operator onto the tangent space of HT,k. In addition, a numerical comparison is made between this dynamical approach and the standard one based on truncated singular value decompositions. © 2013 Elsevier Inc.

Subfields: Computational Mathematics, Computational Mechanics, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mathematics

In Computational Mechanics

In Computational Theory and Mathematics

Citations in the corpus

A literature survey of low‐rank tensor approximation techniques (2013, neutral): “Examples include nonlinear conjugate gradient or Newton-like methods on manifolds of low-rank matrices [35,191,198,257,258] or low-rank tensors [52,53,131,132,255], see also Section 3.4.”
On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format (2013, background): “Our general idea of proof as pursued in this work extends [43] to the (classical and hierarchical) Tucker format and, more generally, to those tensor networks [14] for which the redundancy of the respective parametrization can be characterized explicitly as exemplified in this work for the TT…”
On the approximation of high-dimensional differential equations in the hierarchical Tucker format (2013, methodology): “The same result has also been obtained with other techniques in [12] and [14].”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, unknown): “1 Hierarchical Tucker (HT) and Tensor Tree Network State (TTNS) Models Hierarchical Tucker (HT) decompositions (also called hierarchical tensor representation) have been introduced in [113] and also independently in [106], see also [108, 158, 181, 263] and references therein8.”
Low-rank methods for high-dimensional approximation and model order reduction (2015, unknown): “ J(w), which is a strongly convex quadratic functional. 9This stationarity condition reveals the importance of the analysis of the manifold structure of subsets of tensors with bounded rank (see e.g. [85, 38]). 56 Parameter-dependent equations. We now consider the particular case of the solution of ”
Zur Theorie der Niedrigrangapproximation in Tensorprodukten von Hilberträumen (2013, background): “Die Darstellung ist alles andere als erschöpfend, sondern orientiert sich lose an den im Literaturverzeichnis gelisteten Arbeiten des Autors.”
Low-Rank Tensor Learning with Discriminant Analysis for Action Classification and Image Recovery (2014, background): “…representation learning approaches have been proposed (Chen and Saad 2009; Koch and Lubich 2010; Haegeman et al. 2011; Holtz, Rohwedder, and Schneider 2012; Khoromskij, Oseledets, and Schneider 2012; Arnold and Jahnke 2012; Lubich et al. 2013; Uschmajew and Vandereycken 2013; Mu et al. 2013).”
Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis (2014, background): “Examples of tensor networks are the hierarchical Tucker (HT) decompositions and Tensor Trains (TT) (see Figure 9) [15, 16, 90].”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives (2017, methodology): “For example, [Uschmajew and Vandereycken, 2013] developed the manifold structure for the HT tensors, while Lubich et al.”
Low-rank tensor completion by Riemannian optimization (2014, unknown): “Hence, the chain rule yields the smoothness of the operator PHOr on D. 2.2 Manifold setting The set Mr of tensors of fixed multilinear rank r = (r1, r2, . . . , rd) forms a smooth embedded submanifold of Rn1×···×nd [28, 29].”