Matrix Backpropagation for Deep Networks with Structured Layers

By Catalin Ionescu, Orestis Vantzos, Cristian Sminchisescu. 2015

317 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “ slows the forward and gradient computations of the entire network. Skipping this step allows us to efﬁciently ﬁne-tune the model. We also note that concurrent to our publication [37], Ionescu et al. [28] proposed a DeepO2P approach and noted similar difﬁculties. 4 IMAGE CLASSIFICATION EXPERIMENTS W”
Higher Order Conditional Random Fields in Deep Neural Networks (2015, neutral): “nd-to-end trainable CNN. GraphicalmodelshavebeenusedwithCNNsinotherareasbesidessemanticsegmentation, such as in pose-estimation [29] and group activity recognition [30]. Alternatively, Ionescu et al. [31] incorporated structure into a deep network with structured matrix layers and matrix backpropaga”
Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs (2016, background): “A dominant research direction for improving semantic segmentation with deep learning is the combination of the powerful classification capabilities of FCNs with structured prediction [1,2,3,9,10,11], which aims at improving classification by capturing interactions between predicted labels.”
Bilinear CNNs for Fine-grained Visual Recognition (2015, result): “[28] proposed a DeepO2P approach and noted similar difficulties.”
Learning Monocular 3D Human Pose Estimation from Multi-view Images (2018, unknown): “During training, we do not backpropagate the gradient through this rotation matrix computation, which would require a singular value decomposition and is intricate to differentiate [10].”
Spectral Inference Networks: Unifying Deep and Spectral Learning (2018, background): “In (Ionescu et al., 2015) spectral decompositions were incorporated as differentiable layers in deep network architectures.”
Learning to Drop: Robust Graph Neural Network via Topological Denoising (2020, unknown): “of graph adjacency matrix A�. Singular value decomposition (SVD) is required to optimize the nuclear norm regularization. However, SVD may lead to unstable results during backpropagation. As shown in [21], the partial derivatives of the nuclear norm requires computing of a matrix M� with elements ��”
A Riemannian Network for SPD Matrix Learning (2016, methodology): “While (Ionescu, Vantzos, and Sminchisescu 2015) introduced a covariance pooling layer into ConvNets starting from images, our SPDNet works on SPD matrices directly and exploits multiple layers tailored for SPD matrix deep learning.”
Grassmann Pooling as Compact Homogeneous Bilinear Pooling for Fine-Grained Visual Classification (2018, methodology): “[42] have explored such matrix backpropagation in CNN which is also adopted in this work.”
Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning (2018, methodology): “To minimize the angular distance (4), we backpropagate through the SVD operator using matrix calculus [22, 10].”
Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization (2017, methodology): “G2DeNet [32] is concerned with inserting global Gaussian distributions into ConvNets for end-to-end learning.”
Global Second-Order Pooling Convolutional Networks (2018, methodology): “The pioneering works, i.e., DeepO2P [18] and bilinear CNN (B-CNN) [26], performed global secondorder pooling, rather than the commonly used global average (i.e., first-order) pooling (GAvP) [25], after the last convolutional layers in an end-to-end manner.”
Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (2021, methodology): “From a practical perspective, the gradient and backpropagation through the SVD operation was analyzed by (Ionescu et al., 2015; Papadopoulo and Lourakis, 2000) and implemented in the automatic differentiation frameworks such as PyTorch.”
Attention-Aware Deep Reinforcement Learning for Video Face Recognition (2017, methodology): “We used the results of PittPatt presented in [43] and the results of DeepO2P [22] and VGGFace presented in [18].”
Learning to Find Good Correspondences (2017, background): “Since X diag (w)X is symmetric, the function g that computes Vec (E), and therefore E, from X is a self-adjoint eigendecomposition, which has differentiable closed-form solution with respect to w [15].”
Spectral Inference Networks: Unifying Spectral Methods With Deep Learning (2018, background): “In [18] spectral decompositions were incorporated as differentiable layers in deep network architectures.”