The ubiquitous Kronecker product

By Charles F. Van Loan. 2000

1,027 citations recorded by the source corpus.

Abstract The Kronecker product has a rich and very pleasing algebra that supports a wide range of fast, elegant, and practical algorithms. Several trends in scientific computing suggest that this important matrix operation will have an increasingly greater role to play in the future. First, the application areas where Kronecker products abound are all thriving. These include signal processing, image processing, semidefinite programming, and quantum computing. Second, sparse factorizations and Kronecker products are proving to be a very effective way to look at fast linear transforms. Researchers have taken the Kronecker methodology as developed for the fast Fourier transform and used it to build exciting alternatives. Third, as computers get more powerful, researchers are more willing to entertain problems of high dimension and this leads to Kronecker products whenever low-dimension techniques are “tensored” together.

Subfields: Computational Mechanics, Computational Theory and Mathematics, Numerical Analysis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computational Theory and Mathematics

In Numerical Analysis

Citations in the corpus

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes (2018, methodology): “Each block corresponds to a unique layer in the network, and each block is further approximated as a Kronecker product (see Van Loan, 2000) of two much smaller matrices by assuming that the second-order statistics of the input activation and the back-propagated derivatives within a layer are indepen”
New Insights and Perspectives on the Natural Gradient Method (2014, background): “See Van Loan (2000) for a discussion of Kronecker products and their properties.”
Structured data-sparse approximation to high order tensors arising from the deterministic Boltzmann equation (2007, neutral): “Let us introduce the product index set I (cid:1) = I 1 ⊗· · ·⊗I , where we use multi-indices i (cid:1) = ( i (cid:1), 1 , . . . , i (cid:1),p ) ∈ I (cid:1) , (cid:2) = 1 , . . . , q , with the components i (cid:1),m ∈ { 1 , . . . , N I } , for m = 1 , . . . , p .”
Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication (2005, background): “Proof. Let G1 have the eigenvalues λ1 ,λ 2 ,...,λ N1 . By properties of the Kronecker multiplication [ 19 , 17], the eigenvalues of Gk are kth Kronecker power of”
Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification (2013, methodology): “Cholesky or QR) of a matrix formed by Kronecker products is given by the Kronecker products of the factorizations of the individual matrices [27].”
Passivity enforcement via perturbation of Hamiltonian matrices (2004, methodology): “The Kronecker matrix product [22] will be a useful tool for the formulation of the passivity compensation algorithm.”
Separable approximations of space‐time covariance matrices (2007, background): “Hence, we have the separable approximation � ≈ S1 ⊗ S2 ⊗ T . Unfortunately, a direct minimization of � � − S1 ⊗ S2 ⊗ T � F does not have a closed-form SVD solution, see Van Loan (2000) .”
Passivity Enforcement With Relative Error Control (2007, background): “In this expression, denotes the desired perturbation on the th imaginary eigenvalue, is the Kronecker matrix product [31], [32], matrix is defined as”
Optimizing Neural Networks with Kronecker-factored Approximate Curvature (2015, background): “See Van Loan (2000) for a good discussion of the Kronecker product.”
Low Complexity Iterative Rake Decision Feedback Equalizer for Zero-Padded OTFS Systems (2020, background): “Such permutation is known in the literature as a perfect shuffle, and has the following property [17]: given square matrices A and B”
Network Decoupling: From Regular to Depthwise Separable Convolutions (2018, methodology): “This problem is similar to the Kronecker product decomposition problem [16, 23], which factorizes a tensor into a linear combination of tensor/matrix Kronecker products.”
Kronecker-factored Curvature Approximations for Recurrent Neural Networks (2018, background): “(See Van Loan (2000) for a good discussion of the Kronecker product and its properties.)”
Multilinear operators for higher-order decompositions (2006, background): “1 (Kronecker product [44]) Let A ∈ RI×J ,B ∈ RK×L.”