Beyond Covariance: Feature Representation with Nonlinear Kernel Matrices

By Lei Wang, Jianjia Zhang, Luping Zhou, Chang-Fu Tang, Wanqing Li. 2015

99 citations recorded by the source corpus.

Covariance matrix has recently received increasing attention in computer vision by leveraging Riemannian geometry of symmetric positive-definite (SPD) matrices. Originally proposed as a region descriptor, it has now been used as a generic representation in various recognition tasks. However, covariance matrix has shortcomings such as being prone to be singular, limited capability in modeling complicated feature relationship, and having a fixed form of representation. This paper argues that more appropriate SPD-matrix-based representations shall be explored to achieve better recognition. It proposes an open framework to use the kernel matrix over feature dimensions as a generic representation and discusses its properties and advantages. The proposed framework significantly elevates covariance representation to the unlimited opportunities provided by this new representation. Experimental study shows that this representation consistently outperforms its covariance counterpart on various visual recognition tasks. In particular, it achieves significant improvement on skeleton-based human action recognition, demonstrating the state-of-the-art performance over both the covariance and the existing non-covariance representations.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “Ker-RP [221], lie group manifold [212], orientation matrix [126], temporal relational features [154], node feature map [119].”
When Kernel Methods Meet Feature Learning: Log-Covariance Network for Action Recognition From Skeletal Data (2017, methodology): “We considered the kernel networks of [37] where trialspecific Gram matrix are fed in a second-level kernel responsible for ultimate classification.”
Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, neutral): “[48] 96.”
Kernelized covariance for action recognition (2016, methodology): “TABLE I COMPARATIVE PERFORMANCE OF THE PROPOSED KERNELIZED-COV BENCHMARKING PREVIOUS METHODS IN THE LITERATURE [23], [16], [29] BASED ON COVARIANCE MATRIX AND THE STATE-OF-THE-ART APPROACH [17].”
Distribution Consistency Based Covariance Metric Networks for Few-Shot Learning (2019, background): “…(Tuzel, Porikli, and Meer 2006; 2007; Tabia et al. 2014) or a general representation (Wang et al. 2012; Harandi, Salzmann, and Porikli 2014; Wang et al. 2015b; 2015a; Huang et al. 2015; Wang et al. 2017), due to its favorable properties, e.g. , second-order statistic, symmetric…”
Learning Rich Part Hierarchies With Progressive Attention Networks for Fine-Grained Image Recognition (2020, background): “Feature representation is well studied in image recognition, segmentation and detection fields [34]–[37].”
Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition (2018, methodology): “Large Scale Combined (LSC) This dataset combines nine publicly available datasets (Li, Zhang, and Liu 2010; Wang et al. 2015b; 2016b; Xia, Chen, and”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “The covariance matrix based representation [139], [140] is a representative higher-order (i.”