Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery

By S. Azimi, E. Vig, Reza Bahmanyar, Marco Körner, Peter Reinartz. Asian Conference on Computer Vision, 2018

328 citations recorded by the source corpus.

Automatic multi-class object detection in remote sensing images in unconstrained scenarios is of high interest for several applications including traffic monitoring and disaster management. The huge variation in object scale, orientation, category, and complex backgrounds, as well as the different camera sensors pose great challenges for current algorithms. In this work, we propose a new method consisting of a novel joint image cascade and feature pyramid network with multi-size convolution kernels to extract multi-scale strong and weak semantic features. These features are fed into rotation-based region proposal and region of interest networks to produce object detections. Finally, rotational non-maximum suppression is applied to remove redundant detections. During training, we minimize joint horizontal and oriented bounding box loss functions, as well as a novel loss that enforces oriented boxes to be rectangular. Our method achieves 68.16% mAP on horizontal and 72.45% mAP on oriented bounding box detection tasks on the challenging DOTA dataset, outperforming all published methods by a large margin (\(+6\)% and \(+12\)% absolute improvement, respectively). Furthermore, it generalizes to two other datasets, NWPU VHR-10 and UCAS-AOD, and achieves competitive results with the baselines even when trained on DOTA. Our method can be deployed in multi-class object detection applications, regardless of the image and object scales and orientations, making it a great choice for unconstrained aerial and satellite imagery.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/cc5e5e31556531d5e1078418b45b95d11565db7d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Junchi Yan
2. Xue Yang
3. Guisong Xia
4. Qi Tian
5. Dacheng Tao
6. Jian Ding
7. Luc Van Gool
8. Junwei Han

In Media Technology

1. Guisong Xia
2. Xue Yang
3. Junchi Yan
4. Jian Ding
5. Jiaming Han
6. Xiang Bai
7. Mingkun Yang
8. Qi Tian

Citations in the corpus

• ReDet: A Rotation-equivariant Detector for Aerial Object Detection (2021, methodology): “With the development of general object detection, many well-designed methods have been proposed for oriented object detection [1, 7, 24, 35, 38, 40, 42], showing promising performance on challenging datasets [21, 35].”
• SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects (2018, methodology): “Besides the official baseline given by DOTA, we also compare with RRPN [29], R(2)CNN [19], R-DFPN [41], ICN [2] and RoI-Transformer [8], which are all applicable to multi-category rotation object detection.”
• Gliding Vertex on the Horizontal Bounding Box for Multi-Oriented Object Detection (2019, unknown): “[26] adopt an image-cascade network to extract multi-scale features.”
• Learning RoI Transformer for Oriented Object Detection in Aerial Images (2019, methodology): “The dresnet101 in ICN [1] means deformable conv. resnet101.”
• Oriented RepPoints for Aerial Object Detection (2021, methodology): “2) Quadrilateral-based OBBs: The quadrilateral-based approaches employ four vertices of the quadrilateral to represent the oriented objects [7], [24], [25].”
• Dynamic Anchor Learning for Arbitrary-Oriented Object Detection (2020, unknown): “ICN(Azimi et al. 2018) R-101 81.”
• Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges (2021, methodology): “82 HBB Results method backbone Plane BD Bridge GTF SV LV Ship TC BC ST SBF RA Harbor SP HC mAP ICN [19] DR-101-FPN 89.”
• SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing (2020, methodology): “01 ICN [23] ResNet101 81.”
• Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence (2021, methodology): “In line with [30, 36], we randomly select 1,110 for training and 400 for testing.”
• Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss (2021, methodology): “For aerial images, ICN [2], ROI-Transformer [8], SCRDet [69] and Gliding Vertex [62] are two-stage representative methods whose pipeline comprises of object localization and classification, while DRN [41], R(3)Det [67] and RSDet [42] are singlestage methods.”
• CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery (2019, background): “In recent years, CNNs have been exploited for object detection in remote sensing images, and promising results have been obtained [19], [28]–[32].”
• Dense Label Encoding for Boundary Discontinuity Free Rotation Detection (2020, methodology): “Many rotation detectors (including those for aerial image [46, 15, 47, 2, 6, 51, 45, 30, 48, 49, 50], scene text [56, 23, 14, 26, 19, 18] and face [34, 13, 33]) are derived based on the vanilla detection algorithms [10, 32, 31, 5, 20, 21].”
• Learning Modulated Loss for Rotated Object Detection (2019, methodology): “…i) traditional deep learning meth-ods, such as Faster RCNN [29] and RetinaNet [35]; ii) scene text detection methods, like R 2 CNN [20] and RRPN [21]; iii) recently published methods for multi-category rotation detectors, includes ICN [43], RoI Transformer [44], SCRDet [45] and R 3 Det [10].”
• Dynamic Refinement Network for Oriented and Densely Packed Object Detection (2020, unknown): “6.58 23.42 8.85 2.09 4.82 44.34 38.35 34.65 16.02 37.62 47.23 25.5 7.45 21.39 FR-O [40] 79.42 44.13 17.7 64.05 35.3 38.02 37.16 89.41 69.64 59.28 50.3 52.91 47.89 47.4 46.3 54.13 two-stage method ICN [1] 81.40 74.30 47.70 70.30 64.90 67.80 70.00 90.80 79.10 78.20 53.60 62.90 67.00 64.20 50.20 68.20 ”