Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?

By Otávio Augusto Bizetto Penatti, Keiller Nogueira, J. A. D. Santos. 2015

705 citations recorded by the source corpus.

In this paper, we evaluate the generalization power of deep features (ConvNets) in two new scenarios: aerial and remote sensing image classification. We evaluate experimentally ConvNets trained for recognizing everyday objects for the classification of aerial and remote sensing images. ConvNets obtained the best results for aerial images, while for remote sensing, they performed well but were outperformed by low-level color descriptors, such as BIC. We also present a correlation analysis, showing the potential for combining/fusing different ConvNets with other descriptors or even for combining multiple ConvNets. A preliminary set of experiments fusing ConvNets obtains state-of-the-art results for the well-known UCMerced dataset.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Junwei Han
2. Gong Cheng
3. Liangpei Zhang
4. Xiaoqiang Lu
5. Dacheng Tao
6. Bo Du
7. Guisong Xia
8. Xuelong Li

In Media Technology

1. Liangpei Zhang
2. Gong Cheng
3. Junwei Han
4. Xiaoqiang Lu
5. Bo Du
6. Zhenwei Shi
7. Guisong Xia
8. Gui-Song Xia

Citations in the corpus

• EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification (2017, methodology): “[20], analyzed remotely sensed satellite images with a spatial resolution of 10 m/pixel to classify coffee crops.”
• Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models (2018, background): “…per image tile, thus tackling what in computer vision is called the image classification problem1: authors in Castelluccio et al. (2015) and Penatti et al. (2015) studied the effect of fine-tuning models trained on natural image classification problems, in order to adapt them quickly to…”
• Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis (2020, methodology): “In recent years, some pure supervised DL methods have been suggested for change detection in RS images relying on CNNs [46], [71].”
• Scene classification using multi-scale deeply described visual words (2016, background): “The learnt deep features are e ff ective for describing complex objects and for further image understanding (Penatti, Nogueira, and dos Santos 2015).”
• Deep learning for remote sensing image classification: A survey (2018, methodology): “For example, work in Castelluccio, Poggi, Sansone, and Verdoliva (2015), Penatti et al. (2015), and Xia et al. (2016) directly makes use of features extracted from the fully connected layers as the input of the classifier.”
• Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks (2017, neutral): “vehicle detection [1], semantic labeling [2, 3, 4, 5] and land cover/use classification [6, 7].”
• Land Use Classification in Remote Sensing Images by Convolutional Neural Networks (2015, methodology): “The Brazilian Coffee Scenes dataset [8], instead, includes satellite images with an infra-red band, hence less similar to general purpose images.”
• Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art (2016, methodology): “In the architecture of the conventional CNNs, the final layer is followed by some fully connected layers and finally the classification layer.”
• Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning (2016, methodology): “In contrast, to avoid overfitting problems in training CNN with limited data, the work in [40] investigated the use of the CNN model pretrained in general computer vision data sets for classifying remote sensing scenes.”
• Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification (2017, methodology): “[43], [47] demonstrate consistently using pre-trained CNN models can achieve better results via fine-tuning, which has been successfully used in some studies [38], [5], [34], [44].”
• Exploring Models and Data for Remote Sensing Image Caption Generation (2017, background): “However, the studies of the remote sensing images are still concentrate on scene classification [5], [16], [17], [18], [19], object recognition [20], [21] and segmentation [22], [23], [24], [25].”
• Towards better exploiting convolutional neural networks for remote sensing scene classification (2016, methodology): “However, for small datasets even data augmentation is not enough to avoid overfitting.”
• SEN12MS - A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion (2019, background): “or aerial imagery (R, G, B, NIR) 6 land cover classes over continental USA (Basu et al., 2015) Brazilian Coffee Scenes Dataset 51;004 64 64 SPOT 2 imbalanced classes describing non-coffee and coffee (Penatti et al., 2015) USGS SIRI-WHU 1 10;000 9;000 Color aerial image 4 classes over Montgomery Coun”
• Unmanned Aerial Vehicle for Remote Sensing Applications - A Review (2019, background): “have shown experimentally that deep features obtained by training on everyday objects can be used for the classification of aerial images [116].”