Towards better exploiting convolutional neural networks for remote sensing scene classification

By Keiller Nogueira, Otávio Augusto Bizetto Penatti, J. A. D. Santos. 2016

875 citations recorded by the source corpus.

Abstract We present an analysis of three possible strategies for exploiting the power of existing convolutional neural networks (ConvNets or CNNs) in different scenarios from the ones they were trained: full training, fine tuning, and using ConvNets as feature extractors. In many applications, especially including remote sensing, it is not feasible to fully design and train a new ConvNet, as this usually requires a considerable amount of labeled data and demands high computational costs. Therefore, it is important to understand how to better use existing ConvNets. We perform experiments with six popular ConvNets using three remote sensing datasets. We also compare ConvNets in each strategy with existing descriptors and with state-of-the-art baselines. Results point that fine tuning tends to be the best performing strategy. In fact, using the features from the fine-tuned ConvNet with linear SVM obtains the best results. We also achieved state-of-the-art results for the three datasets used.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification (2017, methodology): “As shown in previous work [6], [15], [17], [19], deep CNNs have been shown to outperform non-deep learning approaches in land use and land cover image classification.”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, methodology): “For example, publicly available trained networks such as GoogLeNet (Szegedy et al., 2015) can be transferred for scene classification for satellite images (Hu et al., 2015; Marmanis et al., 2016; Nogueira et al., 2017).”
Joint Deep Learning for land cover and land use classification (2019, background): “Deep learning methods have achieved huge success not only in classical computer vision tasks, such as target detection, visual recognition and robotics, but also in many other practical applications (Hu et al., 2015; Nogueira et al., 2017).”
Deep learning for remote sensing image classification: A survey (2018, methodology): “Indeed, as shown in Nogueira et al. (2016), using the existing aerial scene data sets (e.g., UC-Merced dataset and WHURS19 data set) to fully train popular CNN architectures, for example, AlexNet (Hu, Xia, et al., 2015), CaffeNet (Jia et al., 2014) or GoogLeNet (Szegedy et al., 2014), may well lead…”
Training Small Networks for Scene Classification of Remote Sensing Images via Knowledge Distillation (2018, background): “A common use of transfer learning in the field of remote sensing is to fine-tune an ImageNet-pretrained model on a remote sensing dataset [2,40,73].”
Scene Classification With Recurrent Attention of VHR Remote Sensing Images (2019, methodology): “Under these circumstances, pretrained CNNs have been transferred to this task [7], [26], [39], which means training the CNN with other lager data sets, e.”
Deep Unsupervised Embedding for Remotely Sensed Images Based on Spatially Augmented Momentum Contrast (2020, background): “Despite its remarkable success [29]–[32], the existing limitations on the number of spectral bands and the data complexity make this transfer learning scheme unable to fully exploit the advantages of RS imagery [33].”
In-domain representation learning for remote sensing (2019, methodology): “This is a relatively small datasets that has been widely benchmarked for remote sensing scene classification task since 2010 and for which nearly perfect accuracy can be achieved with modern convolutional neural networks (Castelluccio et al., 2015; Marmanis et al., 2016; Nogueira et al., 2017).”
Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images (2019, background): “Such feature representation, known as deep features, has exhibited stronger domain invariance capability than shallow features representation for remote sensing classification task [41], [42].”
Deep Learning for Land Use and Land Cover Classification Based on Hyperspectral and Multispectral Earth Observation Data: A Review (2020, background): “In [8,112], authors have compared the mentioned models in the context of land cover classification of HSI.”
The color out of space: learning self-supervised representations for Earth Observation imagery (2020, background): “In these contexts, ﬁne-tuning pre-trained models often mitigate the lack of a large annotated dataset, yielding great performance in some cases [19], [20].”
Deep Feature Fusion for VHR Remote Sensing Scene Classification (2017, methodology): “tuned GoogLe-Net followed by linear SVM introduced in [18], the pretrained Overfeat model followed by CNN classifier”