Multi-scale object detection in remote sensing imagery with convolutional neural networks

By Zhipeng Deng, Hao Sun, Juanping Zhao, Lin Lei, H. Zou. Isprs Journal of Photogrammetry and Remote Sensing, 2018

469 citations recorded by the source corpus.

Abstract Automatic detection of multi-class objects in remote sensing images is a fundamental but challenging problem faced for remote sensing image analysis. Traditional methods are based on hand-crafted or shallow-learning-based features with limited representation power. Recently, deep learning algorithms, especially Faster region based convolutional neural networks (FRCN), has shown their much stronger detection power in computer vision field. However, several challenges limit the applications of FRCN in multi-class objects detection from remote sensing images: (1) Objects often appear at very different scales in remote sensing images, and FRCN with a fixed receptive field cannot match the scale variability of different objects; (2) Objects in large-scale remote sensing images are relatively small in size and densely peaked, and FRCN has poor localization performance with small objects; (3) Manual annotation is generally expensive and the available manual annotation of objects for training FRCN are not sufficient in number. To address these problems, this paper proposes a unified and effective method for simultaneously detecting multi-class objects in remote sensing images with large scales variability. Firstly, we redesign the feature extractor by adopting Concatenated ReLU and Inception module, which can increases the variety of receptive field size. Then, the detection is preformed by two sub-networks: a multi-scale object proposal network (MS-OPN) for object-like region generation from several intermediate layers, whose receptive fields match different object scales, and an accurate object detection network (AODN) for object detection based on fused feature maps, which combines several feature maps that enables small and densely packed objects to produce stronger response. For large-scale remote sensing images with limited manual annotations, we use cropped image blocks for training and augment them with re-scalings and rotations. The quantitative comparison results on the challenging NWPU VHR-10 data set, aircraft data set, Aerial-Vehicle data set and SAR-Ship data set show that our method is more accurate than existing algorithms and is effective for multi-modal remote sensing images.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/7f66e47f397e3b2ade8d3c0df416358d094e441d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Qian Du
2. Wei Li
3. Danfeng Hong
4. Xin Wu
5. Hao Su
6. Ran Tao
7. Feng Xu
8. Jun Shi

In Media Technology

1. Qian Du
2. Wei Li
3. Danfeng Hong
4. Xin Wu
5. Feng Xu
6. Peijin Wang
7. Qingyang Li
8. Ruiping Wang

Citations in the corpus

• Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark (2022, unknown): “, 2018), Multi-Scale CNN (MSCNN) (Deng et al., 2018) and Deep Feature Pyramid Network (DFPN) (Yang et al.”
• Optical Remote Sensing Image Change Detection Based on Attention Mechanism and Image Difference (2021, background): “Many scholars have introduced deep learning technology to remote sensing processing and have achieved excellent performance in image classification [25], [26], semantic segmentation [27]–[29], object detection [30], [31], change detection [17], [32]–[35], and other tasks.”
• SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery (2022, background): “Moreover, the scale of those objects is diverse with multiple categories [11].”
• HRSID: A High-Resolution SAR Images Dataset for Ship Detection and Instance Segmentation (2020, background): “As the high-resolution and vast extent characteristics of SAR imagery, ship detection with SAR imagery has a unique advantage on marine traffic safety monitoring and marine resources development compared to other remote sensing methods [1]–[4].”
• FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery (2021, background): “It is one of the most fundamental and important tasks in the field of earth observation, which serves various civil applications, such as geographic information sys-tem mapping, agriculture, traffic planning, and navigation [53, 52, 19, 50, 49, 7, 6, 11].”
• Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey (2021, methodology): “The most commonly used is the multiscale feature map [23], which is the output of multiple filters on multiple feature maps or multiple filters on a single feature map [22], [24], [50]–[59].”
• UAV-YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective (2020, background): “Object detection can be divided into traditional handcrafted feature-based object detection [9,10] and deep-learning-based object detection [11,12].”