A Riemannian Network for SPD Matrix Learning

By Zhiwu Huang, Luc Van Gool. 2016

523 citations recorded by the source corpus.

Symmetric Positive Definite (SPD) matrix learning methods have become popular in many image and video processing tasks, thanks to their ability to learn appropriate statistical representations while respecting Riemannian geometry of underlying SPD manifolds. In this paper we build a Riemannian network architecture to open up a new direction of SPD matrix non-linear learning in a deep model. In particular, we devise bilinear mapping layers to transform input SPD matrices to more desirable SPD matrices, exploit eigenvalue rectification layers to apply a non-linear activation function to the new SPD matrices, and design an eigenvalue logarithm layer to perform Riemannian computing on the resulting SPD matrices for regular output layers. For training the proposed deep network, we exploit a new backpropagation with a variant of stochastic gradient descent on Stiefel manifolds to update the structured connection weights and the involved SPD matrix data. We show through experiments that the proposed SPD matrix network can be simply trained and outperform existing SPD matrix learning and state-of-the-art methods in three typical visual classification tasks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Jie Zhou
3. Jiwen Lu
4. Yongming Rao
5. Yi Yang
6. Xiaogang Wang
7. Dacheng Tao
8. Chen Change Loy

Citations in the corpus

• When Kernel Methods Meet Feature Learning: Log-Covariance Network for Action Recognition From Skeletal Data (2017, methodology): “Last, but not least, in all works [5, 32, 38, 21, 19, 14, 15] the network adopted are deep, which actually makes the overall pipeline difficult to train, since overfitting must be controlled, the problem of local minima and saddle points must be faced, massive training data is required, as well as e”
• Deep Learning on Lie Groups for Skeleton-Based Action Recognition (2016, methodology): “Moreover, recently some deep learning models have emerged [10, 7, 28, 25, 18, 21] that deal with data in a nonEuclidean domain.”
• Attention-Aware Deep Reinforcement Learning for Video Face Recognition (2017, methodology): “We used the results of PittPatt presented in [43] and the results of DeepO2P [22] and VGGFace presented in [18].”
• Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition (2018, methodology): “Secondly, to fairly compare the current deep learning methods, we conduct 10 random evaluations, each of which randomly selects half of the sequences for training and the rest for testing (Huang and Van Gool 2017).”
• Federated Transfer Learning for EEG Signal Classification (2020, methodology): “For example, the Riemannian network architecture designed by Huang and Goal that includes BiMap layer, ReEig layer and LogEig layer on SPD matrices, has outperformed the existing state-of-the-art methods in three typical visual classification tasks [14].”
• Building Deep Networks on Grassmann Manifolds (2016, methodology): “Training the GrNet per epoch (100 Method AFEW HDM05 STM-ExpLet [41] 31.73% – DeepO2P [33] 28.54% – RSR-SPDML [21] – 48.01% ± 3.38 SPDNet [29] 34.23 % 61.45% ± 1.12 DCC [39] 25.78% 41.74% ± 1.92 GDA [20] 29.11% 46.25% ± 2.71 GGDA [19] 29.45% 46.87% ± 2.19 PML [31] 28.98% epoches in total) costs…”