A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection

By X. Wang, Abhinav Shrivastava, Abhinav Kumar Gupta. 2017

612 citations recorded by the source corpus.

How do we learn an object detector that is invariant to occlusions and deformations? Our current solution is to use a data-driven strategy – collect large-scale datasets which have object instances under different conditions. The hope is that the final classifier can use these examples to learn invariances. But is it really possible to see all the occlusions in a dataset? We argue that like categories, occlusions and object deformations also follow a long-tail. Some occlusions and deformations are so rare that they hardly happen, yet we want to learn a model invariant to such occurrences. In this paper, we propose an alternative solution. We propose to learn an adversarial network that generates examples with occlusions and deformations. The goal of the adversary is to generate examples that are difficult for the object detector to classify. In our framework both the original detector and adversary are learned in a joint manner. Our experimental results indicate a 2.3% mAP boost on VOC07 and a 2.6% mAP boost on VOC2012 object detection challenge compared to the Fast-RCNN pipeline.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
Random Erasing Data Augmentation (2017, methodology): “The recent A-Fast-RCNN [24] learns an occlusion invariant object detector by training an adversarial network that generates examples with occlusion.”
Adversarial AutoAugment (2019, background): “This is because that non-differentiable augmentation operations break gradient flow from the target network F to the augmentation policy network A (Wang et al., 2017; Peng et al., 2018).”
DropBlock: A regularization method for convolutional networks (2018, background): “DropBlock is different from recent work [32] which learns to drop a structured pattern on features of region proposals.”
Single-Shot Refinement Neural Network for Object Detection (2017, methodology): “After that, numerous effective techniques are proposed to further improve the performance, such as architecture diagram [5, 25, 55], training strategy [31, 41, 48, 50], contextual reasoning [1, 14, 40, 51] and multiple layers exploiting [3, 24, 26, 42].”
Learning Data Augmentation Strategies for Object Detection (2019, background): “In the same vein, [48] learns an occlusion pattern for each object to create adversarial examples.”
Seeing What a GAN Cannot Generate (2019, background): “Generative Adversarial Networks [15] have enabled many computer vision and graphics applications such as generation [7, 21, 22], image and video manipulation [19, 20, 30, 35, 39, 41, 52], object recognition [6, 42], and text-to-image translation [33, 45, 49].”
RODEO: Replay for Online Object Detection (2020, neutral): “Following [51], we use the Fast RCNN architecture [55] with a ResNet-50 [18] backbone and edge box object proposals [59] for all models, unless otherwise noted.”
Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving (2019, background): “Learning a model invariant to such occurrences is proposed in [42] where an adversarial network learns to generate challenging occlusions and deformations examples.”
FILM: Following Instructions in Language with Modular Methods (2021, methodology): “) use the same family of segmentation/ detection models as FILM, such as Mask-RCNN and Fast-RCNN (Wang et al., 2017).”
3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions (2019, background): “GANs for 3D Point Clouds Generation: GANs [13] for 2D image generation tasks have been widely studied with great success [10, 18, 23, 26, 33, 36, 42, 43, 49, 50], but GANs for 3D point cloud generation have rarely been studied in the computer vision field.”
Zoom Out-and-In Network with Map Attention Decision for Region Proposal and Object Detection (2017, background): “(Wang et al., 2017) proposed to learn an adversarial network that generates examples with occlusions and deformations, which are hard for the detector to classify.”
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition (2019, methodology): “…and monitoring the changes at the output of a model has been used to visualizing whether a classiﬁer can localize objects in an image [67], learning object detectors from weak image-level labels [3, 49, 36, 37], improving robustness of object detectors [59], and regularizing model training [18].”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “loped adversarially learned inference (ALI). 2.5.3 Adversarial Loss. Adversarial losses, an idea stemming from GANs, have been applied in multiple settings outside of generative modeling. Wang et al. [121] created an adversarial spacial dropout network to add occlusions to images to improve the accu”
Recent Advances in Deep Learning for Object Detection (2019, methodology): “[159] proposed A-Fast-R-CNN which was 1334 trained by generated adversarial examples.”
VITAL: VIsual Tracking via Adversarial Learning (2018, background): “series of works on network training [1, 40, 18] and computer vision applications, such as image generation [62], image stylization [26], object detection [58], and semantic segmentation [53].”