Learning RoI Transformer for Oriented Object Detection in Aerial Images

By Jian Ding, Nan Xue, Yang Long, Guisong Xia, Qikai Lu. Computer Vision and Pattern Recognition, 2019

1,156 citations recorded by the source corpus.

Object detection in aerial images is an active yet challenging task in computer vision because of the bird’s-eye view perspective, the highly complex backgrounds, and the variant appearances of objects. Especially when detecting densely packed objects in aerial images, methods relying on horizontal proposals for common object detection often introduce mismatches between the Region of Interests (RoIs) and objects. This leads to the common misalignment between the final object classification confidence and localization accuracy. In this paper, we propose a RoI Transformer to address these problems. The core idea of RoI Transformer is to apply spatial transformations on RoIs and learn the transformation parameters under the supervision of oriented bounding box (OBB) annotations. RoI Transformer is with lightweight and can be easily embedded into detectors for oriented object detection. Simply apply the RoI Transformer to light head RCNN has achieved state-of-the-art performances on two common and challenging aerial datasets, i.e., DOTA and HRSC2016, with a neglectable reduction to detection speed. Our RoI Transformer exceeds the deformable Position Sensitive RoI pooling when oriented bounding-box annotations are available. Extensive experiments have also validated the flexibility and effectiveness of our RoI Transformer.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/aeef4a512aecb76ca04f102ec8972fe8dbce71e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Xue Yang
2. Junchi Yan
3. Guisong Xia
4. Wentao Wang
5. Jian Ding
6. Qi Ming
7. Nan Xue
8. Qi Tian

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Junwei Han
3. Junchi Yan
4. Xue Yang
5. Gong Cheng
6. Luc Van Gool
7. Qi Tian
8. Guisong Xia

Citations in the corpus

• Large Selective Kernel Network for Remote Sensing Object Detection (2023, methodology): “The two-stage RoI transformer [12] uses fully-connected layers to rotate candidate horizontal anchor boxes in the first stage, and then features within the boxes are extracted for further regression and classification.”
• Dynamic Refinement Network for Oriented and Densely Packed Object Detection (2020, methodology): “Although DCN [4] and RoI Trans [5] model the offset prediction in a dynamic manner, they do not change the kernel weight.”
• Drone-Based RGB-Infrared Cross-Modality Vehicle Detection Via Uncertainty-Aware Learning (2020, positive): “The experiments on 708 UA-CMDet∗ with CM-E (element-wise-add operation) also709 demonstrated similar findings.710 2) Cross-Modal Fusion: To verify the effectiveness of the711 cross-modal fusion operation in UA-CMDet, we selected712 RoITransformer as the baseline and trained it on the RGB713 modality”
• Object Detection in 20 Years: A Survey (2019, neutral): “Another recent solution is to learn geometric transformations of the object candidates [172], [173], [174], [175].”
• R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object (2019, methodology): “the remote sensing image scene and the large number of small, cluttered and rotated objects, two-stage rotation detectors are still dominant for their robustness. Among them, ICN [2], ROI-Transformer [10] and SCRDet [40] are state-of-the-art detectors. However, they use a more complicated structure ”
• SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery (2022, background): “Currently, most object detection techniques are solely designed and applied for a single modality, such as red-green-blue (RGB) and infrared (IR) [12], [13].”
• Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review-Part I: Evolution and Recent Trends (2020, methodology): “This becomes necessary due to the rotational invariance of objects in Earth observation data, which when not considered can easily lead to a large amount of false negatives when objects are densely accumulated [100,138].”
• Learning Modulated Loss for Rotated Object Detection (2019, methodology): “…i) traditional deep learning meth-ods, such as Faster RCNN [29] and RetinaNet [35]; ii) scene text detection methods, like R 2 CNN [20] and RRPN [21]; iii) recently published methods for multi-category rotation detectors, includes ICN [43], RoI Transformer [44], SCRDet [45] and R 3 Det [10].”
• ReDet: A Rotation-equivariant Detector for Aerial Object Detection (2021, methodology): “, RoI Transformer [7] learns to transform HRoIs to RRoIs and warps region features with a rotated position sensitive RoI Align.”
• Gliding Vertex on the Horizontal Bounding Box for Multi-Oriented Object Detection (2019, methodology): “39 percent mAP, outperforming the state-of-the-art method [5] by 5.”
• Anchor-Free Oriented Proposal Generator for Object Detection (2021, methodology): “RoI Transformer [1] trains RoI Learner module to transform horizontal proposals into oriented proposals, which significantly increases the detection accuracy.”
• FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery (2021, methodology): “We select RetinaNet [31], Faster R-CNN [41], Cascade R-CNN [2], Gliding Vertex [60], and ROI Transformer [13] as our baseline models with a ResNet-101 backbone [22].”
• SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing (2020, background): “ICN [23], ROI Transformer [24], and SCRDet [30] are representative of two-stage aerial images rotation detectors, which are mainly designed from the perspective of feature extraction.”
• Advancing Plain Vision Transformer Toward Remote Sensing Foundation Model (2022, methodology): “Previous methods mostly utilize the ImageNet [41] dataset for pretraining and improve performance in classification, detection, and segmentation with task-specific designs [12], [42], [43].”
• Oriented R-CNN for Object Detection (2021, methodology): “R(·) is a rotation transformation the same as [7].”
• Align Deep Features for Oriented Object Detection (2020, methodology): “Recently, RoI transformer [4] was proposed to address this issue by transforming horizontal RoIs into rotated RoIs, avoiding a large number of anchors, but it still needs heuristically defined anchors and complex RoI operation.”
• Arbitrary-Oriented Object Detection with Circular Smooth Label (2020, methodology): “,44], denotes the acute angle to the x-axis, and for the other side we refer it as w. It should be distinguished from another denition (x;y;h;w;) illustrated in Figure 2(b), with 180angular range [4,27], whose is determined by the long side (h) of the rectangle and x-axis. The eight-parameter regres”
• Poly Kernel Inception Network for Remote Sensing Detection (2024, methodology): “Consequently, numerous prior efforts have been dedicated to developing various oriented bounding box (OBB) detectors [10, 20, 31, 65, 67, 71] and improving the angle prediction accuracy for OBBs [68, 70, 72–74].”