The treasure beneath convolutional layers: Cross-convolutional-layer pooling for image classification

By Lingqiao Liu, Chunhua Shen, Anton van den Hengel. 2014

200 citations recorded by the source corpus.

A number of recent studies have shown that a Deep Convolutional Neural Network (DCNN) pretrained on a large dataset can be adopted as a universal image descriptor, and that doing so leads to impressive performance at a range of image classification tasks. Most of these studies, if not all, adopt activations of the fully-connected layer of a DCNN as the image or region representation and it is believed that convolutional layer activations are less discriminative. This paper, however, advocates that if used appropriately, convolutional layer activations constitute a powerful image representation. This is achieved by adopting a new technique proposed in this paper called cross-convolutional-layer pooling. More specifically, it extracts subarrays of feature maps of one convolutional layer as local features, and pools the extracted features with the guidance of the feature maps of the successive convolutional layer. Compared with existing methods that apply DCNNs in the similar local feature setting, the proposed method avoids the input image style mismatching issue which is usually encountered when applying fully connected layer activations to describe local regions. Also, the proposed method is easier to implement since it is codebook free and does not have any tuning parameters. By applying our method to four popular visual classification tasks, it is demonstrated that the proposed method can achieve comparable or in some cases significantly better performance than existing fully-connected layer based image representations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning (2016, methodology): “Liu et al. (Liu, Shen, and van den Hengel 2015) extracted and visualized a subspace of CNN features.”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “Another recently proposed method that reports strong results on this setting is the “cross-layer pooling” method of [24] that considers pairwise features extracted from two different layers of a CNN.”
Examining CNN representations with respect to Dataset Bias (2017, methodology): “Dataset: We tested the proposed method on the Large-scale CelebFaces Attributes (CelebA) dataset (Liu et al. 2015) and the SUN Attribute database (Patterson et al. 2014).”
Deep learning features at scale for visual place recognition (2017, unknown): “B with two other state-of-the-art convolution feature encoding methods – the cross-layer pooling [33] and the holistic pooling [34].”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, methodology): “[77] developed a cross-layer pooling method by aggregating the part-based deep descriptors using activations from two successive con-”
Convolutional Features for Correlation Filter Based Visual Tracking (2015, methodology): “The work of [29] proposes a cross-convolutional layer pooling approach.”
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking (2016, background): “Recently, the last (deep) convolutional layers were shown to be more beneficial for image classification [8,33].”
Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers (2016, background): “The idea of using intermediate convolutional layers to complement high level convolutional features has also been exploited for image classification and segmentation [22, 15, 23], video event detection [40], image retrieval [2, 25] and edge detection [39].”
Aggregating Local Deep Features for Image Retrieval (2015, background): “Recently, research attention shifted from the features extracted from the fully-connected layers to the features from the deep convolutional layers of CNNs [5, 22, 14] (below we refer to these features as deep convolutional features).”
Low-Rank Bilinear Pooling for Fine-Grained Classification (2016, background): “This provides a partial explanation as to why simple global pooling achieves such good results without an explicit spatial transformer or cross-channel pooling architecture (e.g. [21])”