Tensor Decomposition for Signal Processing and Machine Learning
By N. Sidiropoulos, Lieven De Lathauwer, Xiao Fu, Kejun Huang, Evangelos E. Papalexakis, Christos Faloutsos. IEEE Transactions on Signal Processing, 2016
1,589 citations recorded by the source corpus.
Tensors or <italic>multiway arrays</italic> are functions of three or more indices <inline-formula> <tex-math notation="LaTeX">$(i,j,k,\ldots)$</tex-math></inline-formula>—similar to matrices (two-way arrays), which are functions of two indices <inline-formula><tex-math notation="LaTeX">$(r,c)$</tex-math></inline-formula> for (row, column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining, and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth <italic>and depth</italic> that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.
Subfields: Artificial Intelligence, Computational Mathematics, Signal Processing
View Paper: https://www.semanticscholar.org/paper/eed9fa4483cab37eacd59db0fac4b1441431ee85
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computational Mathematics
- Andrzej S. Cichocki
- Anh Huy Phan
- Qibin Zhao
- Danilo P. Mandic
- Ivan V. Oseledets
- Namgil Lee
- Timothy M. Hospedales
- Boris N. Khoromskij
In Signal Processing
Citations in the corpus
- Tensor Regression Networks (2017, background): “One class of broadly useful techniques within tensor methods are tensor decompositions, which have been studied for a variety of applications in signal processing and machine learning (Sidiropoulos et al., 2017; Cichocki et al., 2015), data mining and fusion (Papalexakis et al.”
- TensorLy: Tensor Learning in Python (2016, background): “The interested reader is referred to several surveys in the topic, which focus range from basics of multilinear (tensor) algebra and overview of different established tensor decompositions [17, 29], to algorithmic advances [7, 19, 10, 6, 26] and applications [1].”
- Lightspeech: Lightweight and Fast Text to Speech with Neural Architecture Search (2021, methodology): “There have been many techniques on designing lightweight and efficient neural networks, such as shrinking, tensor decomposition [10], quantization [11] and pruning [12].”
- HiCOO: Hierarchical Storage of Sparse Tensors (2018, methodology): “Optimizing CPD and the MTTKRP operation has been well-studied recently [25, 44, 45].”
- Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, positive): “6 Scope and Objectives Review and tutorial papers [7, 42, 54, 87, 119, 137, 163, 189] and books [43, 91, 124, 190] dealing with TDs and TNs already exist, however, they typically focus on standard models, with no explicit links to very largescale data processing topics or connections to a wide class”
- Channel Estimation for RIS-Aided Multiuser Millimeter-Wave Systems (2021, background): “Step (a) is due to the property vec ( XDiag (et)Y T ) = (Y ⊙X) et [24].”
- Canonical polyadic decomposition of third-order tensors: relaxed uniqueness conditions and algebraic algorithm (2015, background): “ers. It is its uniqueness properties that make CPD a basic tool for signal separation and data analysis, with many concrete applications in telecommunication, array processing, machine learning, etc. [4, 5, 18, 22]. The contribution of this paper is twofold. First, we nd very mild conditions for uni”
- Factorized Higher-Order CNNs With an Application to Spatio-Temporal Emotion Estimation (2019, methodology): “Tensor methods for efficient deep networks The properties of tensor methods [14, 41, 17] make them a prime choice for deep learning.”