Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery

By Fan Hu, Gui-Song Xia, Jingwen Hu, Liangpei Zhang. 2015

1,145 citations recorded by the source corpus.

Learning efficient image representations is at the core of the scene classification task of remote sensing imagery. The existing methods for solving the scene classification task, based on either feature coding approaches with low-level hand-engineered features or unsupervised feature learning, can only generate mid-level image features with limited representative ability, which essentially prevents them from achieving better performance. Recently, the deep convolutional neural networks (CNNs), which are hierarchical architectures trained on large-scale datasets, have shown astounding performance in object recognition and detection. However, it is still not clear how to use these deep convolutional neural networks for high-resolution remote sensing (HRRS) scene classification. In this paper, we investigate how to transfer features from these successfully pre-trained CNNs for HRRS scene classification. We propose two scenarios for generating image features via extracting CNN features from different layers. In the first scenario, the activation vectors extracted from fully-connected layers are regarded as the final image features; in the second scenario, we extract dense features from the last convolutional layer at multiple scales and then encode the dense features into global image features through commonly used feature coding approaches. Extensive experiments on two public scene classification datasets demonstrate that the image features obtained by the two proposed scenarios, even with a simple linear classifier, can result in remarkable performance and improve the state-of-the-art by a significant margin. The results reveal that the features from pre-trained CNNs generalize well to HRRS datasets and are more expressive than the low- and mid-level features. Moreover, we tentatively combine features extracted from different CNN models for better performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Taskonomy: Disentangling Task Transfer Learning (2018, background): “These representations, however, can transmit statistics useful for solving other outputs (tasks), presumably if the tasks are related in some form [83, 19, 58, 46].”
A survey of deep neural network architectures and their applications (2017, methodology): “In addition, the CNNs have also been used for high-resolution remote sensing (HRRS) scene classification [66] and handwritten Hangul recognition [84].”
RSVQA: Visual Question Answering for Remote Sensing Data (2020, background): “Nowadays, this family of methods is used to tackle a variety of tasks; for scene classification, an early work in [9] evaluated the possibility to adapt networks pretrained on large natural image databases (such as ImageNet [10]) to classify hyperspectral remote sensing images.”
Remote Sensing Image Scene Classification With Self-Supervised Paradigm Under Limited Labeled Samples (2020, background): “Particularly, RSIs scene classification, which aims to classify scene images into different semantic categories, has been a hot topic driven by applications such as land resource management and urban planning [1], [2].”
Hierarchical and Robust Convolutional Neural Network for Very High-Resolution Remote Sensing Object Detection (2019, background): “[3] consider convolutional feature coding of a group of multiscale images to solve the scaling problem in RSI scene classification.”
Joint Deep Learning for land cover and land use classification (2019, background): “Deep learning methods have achieved huge success not only in classical computer vision tasks, such as target detection, visual recognition and robotics, but also in many other practical applications (Hu et al., 2015; Nogueira et al., 2017).”
Deep Learning for Hyperspectral Image Classification: An Overview (2019, background): “In recent years, TL has been successfully applied in many fields, especially in the remote sensing field [114], [115] where the available training samples are not sufficient.”
Very Deep Convolutional Neural Networks for Complex Land Cover Mapping Using Multispectral Remote Sensing Imagery (2018, background): “The other two strategies are more useful when a limited amount of training samples is available [26,32].”
AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification (2016, methodology): “growing attention in recent years [2]–[4], [7]–[42].”
Remote Sensing Image Scene Classification Using CNN-CapsNet (2019, methodology): “As for CNN, two representative CNN models (VGG‐16 and Inception‐V3) fully trained on the ImageNet dataset are used as initial feature map extractors, considering their popularity in the remote sensing field [25,27,28,35,53,56,59].”
Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification (2017, methodology): “The WHU-RS dataset [32], collected from Google Earth (Google Inc.”
Unmanned Aerial Vehicle for Remote Sensing Applications - A Review (2019, background): “In addition, transfer learning techniques [114] are becoming more needed as the traditional sample-per-dataset learning in large-format satellite images are no longer suitable for local scale, as datasets might be diverse and small in terms of geographical regions [115].”
Deep learning in environmental remote sensing: Achievements and challenges (2020, methodology): “Hu et al. (2018b) employed a CNN to AMSR-E brightness temperature images to retrieve the daily global SM.”