Tensor Spaces and Numerical Tensor Calculus

By Wolfgang Hackbusch. Springer Series in Computational Mathematics, 2012

636 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

View Paper: https://www.semanticscholar.org/paper/ad4b64c3b1c9459af5dcd196c9a1c3e7b03cb988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Which transformer architecture fits my data? A vocabulary bottleneck in self-attention (2021, background): “GRID TENSORS PROVIDE LOWER BOUNDS FOR THE SEPARATION RANK We now present the concept of grid tensors, which are a form of function discretization (Hackbusch, 2012).”
Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks (2022, background): “Moreover, for a given finite precision limit of b = O (1) bits, any neural network with input degree of O (1) such as VGG (Simonyan & Zisserman, 2014) and binary function with polynomial hierarchical decomposition ranks (Hackbusch, 2012), are considered efficiently decomposable according to our defi”
On the Expressive Power of Overlapping Architectures of Deep Learning (2017, background): “For a linear transform F, as defined above in eq.6, and a partition P ·∪ Q, if F (1), . . . , F (N) are non-singular matrices, then F is invertible and the matrix rank of JAKP,Q equals to the matrix rank of JF(A)KP,Q (see proof in Hackbusch (2012)).”
Exact Tensor Completion Using t-SVD (2015, background): “This terminology comes from representation of multilinear functionals on the outer product of finite dimensional vector spaces as an indexed array of numbers, [3] as a vector of ranks of the factors in the contracted representation using matrix product states.”
Tensor Decomposition for Signal Processing and Machine Learning (2016, background): “The error ||X− X̂||(2)F is in fact known to be suboptimal by at most a factor 3 (factor N in the N -th order case) [13], [14].”
Limits to Depth Efficiencies of Self-Attention (2020, methodology): “We now present the concept of grid tensors, which are a form of function discretization [Hackbusch, 2012].”
Implicit Regularization in Deep Learning May Not Be Explainable by Norms (2020, background): “5 in [33]), we may conclude Pr(det(W ) > 0) = Pr(det(W ) < 0) = 0.”
Knowledge (2017, background): “This interplay can be expressed only when we distinguish between external and internal complexity, as in Jost (2004). We will now analyze this active structuring process in more detail, particularly in the context of human cognition.”
Matrix product states and projected entangled pair states: Concepts, symmetries, theorems (2020, background): “milar identication with a simpler ansatz, such as linear combinations of polynomially many product states, as it is the case for instance in (Beylkin and Mohlenkamp, 2002; De Lathauwer et al., 2000; Hackbusch, 2012; Kolda and Bader, 2009). Let us prove that this is not possible, since one can show t”
Inductive Bias of Deep Convolutional Networks through Pooling Geometry (2016, background): “3 describes what is called a hierarchical tensor decomposition (see chapter 11 in [12]), with underlying tree over modes being a full quad-tree (corresponding to the fact that the network’s pooling windows cover four entries each).”
Limits to Depth Efﬁciencies of Self-Attention Supplementary Material (2021, neutral): “We now present the concept of grid tensors, which are a form of function discretization [Hackbusch, 2012].”
Tensor networks for complex quantum systems (2018, background): “Later on, it was proven that the state produced by DMRG is a “finitelycorrelated state”, i.e., essentially what we call today Matrix Product State (MPS) [5], or equivalently, what some mathematicians call Tensor Train [6].”