Global Second-Order Pooling Convolutional Networks

By Zilin Gao, Jiangtao Xie, Qilong Wang, P. Li. Computer Vision and Pattern Recognition, 2018

413 citations recorded by the source corpus.

Deep Convolutional Networks (ConvNets) are fundamental to, besides large-scale visual recognition, a lot of vision tasks. As the primary goal of the ConvNets is to characterize complex boundaries of thousands of classes in a high-dimensional space, it is critical to learn higher-order representations for enhancing non-linear modeling capability. Recently, Global Second-order Pooling (GSoP), plugged at the end of networks, has attracted increasing attentions, achieving much better performance than classical, first-order networks in a variety of vision tasks. However, how to effectively introduce higher-order representation in earlier layers for improving non-linear capability of ConvNets is still an open problem. In this paper, we propose a novel network model introducing GSoP across from lower to higher layers for exploiting holistic image information throughout a network. Given an input 3D tensor outputted by some previous convolutional layer, we perform GSoP to obtain a covariance matrix which, after nonlinear transformation, is used for tensor scaling along channel dimension. Similarly, we can perform GSoP along spatial dimension for tensor scaling as well. In this way, we can make full use of the second-order statistics of the holistic image throughout a network. The proposed networks are thoroughly evaluated on large-scale ImageNet-1K, and experiments have shown that they outperform non-trivially the counterparts while achieving state-of-the-art results.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a5f782f08ae8df9b9926200fdc5adbe99565513a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

FcaNet: Frequency Channel Attention Networks (2020, methodology): “Motivated by CBAM, GSoP [11] introduces a second-order pooling method for downsampling.”
Learning Deep Bilinear Transformation for Fine-grained Image Representation (2019, methodology): “Second-order pooling convolutional networks[19] also proposed to integrate bilinear interactions into convolutional blocks, while they only use such bilinear features for weighting convlutional channels.”
Attention mechanisms in computer vision: A survey (2021, methodology): “[54] proposed to improve the squeeze module by using a global second-order pooling (GSoP) block to model high-order statistics while gathering global information.”
Attention Spiking Neural Networks (2022, background): “There are many optimizations from various aspects for the modeling of the channel and spatial attention, such as effectiveness [42], space complexity [30], computational complexity [43], etc.”
Rotate to Attend: Convolutional Triplet Attention Module (2020, methodology): “f learnable parameters, our approach works much better than the baselines and is also comparable to the state-of-the-art methods that need large additional parameters and computations, like GSoP-Net1 [10]. for sample images to showcase the ability of triplet attention to capture more deterministic f”
Pooling Methods in Deep Neural Networks, a Review (2020, positive): “Second-order pooling convolutional networks [55] also proposed to integrate bilinear interactions into convolutional blocks, while they only use such bilinear features for weighting convolutional channels.”
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (2019, methodology): “, feature recalibration) in SENet (Hu, Shen, and Sun 2018), some researches improve SE block by capturing more sophisticated channel-wise dependencies (Woo et al. 2018; Chen et al. 2018; Gao et al. 2019; Fu et al. 2019) or by combining with additional spatial attention (Woo et al.”