Perceptual Generative Adversarial Networks for Small Object Detection

By Jianan Li, Xiaodan Liang, Yunchao Wei, Tingfa Xu, Jiashi Feng, Shuicheng Yan. 2017

839 citations recorded by the source corpus.

Detecting small objects is notoriously challenging due to their low resolution and noisy representation. Existing object detection pipelines usually detect small objects through learning representations of all the objects at multiple scales. However, the performance gain of such ad hoc architectures is usually limited to pay off the computational cost. In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to super-resolved ones, achieving similar characteristics as large objects and thus more discriminative for detection. For this purpose, we propose a new Perceptual Generative Adversarial Network (Perceptual GAN) model that improves small object detection through narrowing representation difference of small objects from the large ones. Specifically, its generator learns to transfer perceived poor representations of the small objects to super-resolved ones that are similar enough to real large objects to fool a competing discriminator. Meanwhile its discriminator competes with the generator to identify the generated representation and imposes an additional perceptual requirement - generated representations of small objects must be beneficial for detection purpose - on the generator. Extensive evaluations on the challenging Tsinghua-Tencent 100K [45] and the Caltech [9] benchmark well demonstrate the superiority of Perceptual GAN in detecting small objects, including traffic signs and pedestrians, over well-established state-of-the-arts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “GANs enable a wide variety of applications such as image generation [1, 41, 60], representation learning [44], image manipulation [62], object detection [32], and video applications [37, 50, 52].”
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? (2019, neutral): “texture synthesis [18, 33, 28], video generation [31, 30], image-to-image translation [11, 36, 1, 24] and object detection [19].”
Semantics Disentangling for Text-To-Image Generation (2019, background): “GAN-based algorithms show excellent performance in image generation [21, 1, 20, 25, 35, 2, 23].”
Image2StyleGAN++: How to Edit the Embedded Images? (2019, background): “object detection [23], texture synthesis [22, 37, 31], image-to-image translation [16, 42, 28, 25] and video generation [33, 32, 35, 34].”
LOHO: Latent Optimization of Hairstyles via Orthogonalization (2021, background): “Generative models, in particular GANs, have been very successful across various computer vision applications such as image-to-image translation [15, 32, 40], video generation [34, 33, 9], and data augmentation for discriminative tasks such as object detection [24].”
Fast Underwater Image Enhancement for Improved Visual Perception (2019, methodology): “Qualitative performance comparison of FUnIE-GAN and FUnIE-GAN-UP with learning-based methods: CycleGAN [17], UGAN-P [6], Pix2Pix [16], LS-GAN [28], Res-GAN [43], and Res-WGAN [26]; the super-scripted asterisk (∗) denotes unpaired training.”
Bidirectional Conditional Generative Adversarial Networks (2017, methodology): “data from complex distributions for a variety of applications such as image editing [23], image synthesis from text descriptions [24], image super-resolution [14], video summarization [17] and others [2,8,10,11,15,26,28,29,30, 32,34,35], with very promising results. The core methodology for modeling”
Improved YOLOv5 network for real-time multi-scale traffic sign detection (2021, background): “A novel perceptual generative adversarial network was developed for the recognition of small-sized traffic signs [18], which boosts detection performance by generating super-resolve representations for small traffic signs.”
Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving (2019, background): “Pixel level [37], GraspGAN [38], [39], [40] Object Detection SeGAN [41], [42], Perceptual-GAN [43] Super Resolution SRGAN [23] Inpainting [44], [45], [46] Advanced Data Augmentation Context-aware Synthesis and Placement of Object Instances [20] domains without supervision.”
Towards Large-Scale Small Object Detection: Survey and Benchmarks (2022, background): “To deal with this issue, Li et al. [121] devised PerceptualGAN to mine and exploit the intrinsic correlations between small-scale and large-scale objects, in which the generator learns to map the weak representations of small objects to super-resolved ones to deceive the discriminator.”
Generative Adversarial Networks (GANs) (2020, methodology): “To handle this issue, [233] proposed a new Perceptual Generative Adversarial Network [234] model for detecting small objects by reducing the representation difference between small objects and large ones.”
A Normalized Gaussian Wasserstein Distance for Tiny Object Detection (2021, neutral): “GAN-based Detectors: Perceptual GAN [12] is the first to attempt to apply GAN to small object detection, it improves small object detection through narrowing representation difference of small objects from the large ones.”
Tiny Object Detection in Aerial Images (2021, background): “Despite the fact that lots of methods tackled this problem [9]–[13], as shown in recent work about the upper bound of object detection performance [14], there is still a large gap between current and the upper bound performances on tiny object detection.”
Revisiting RCNN: On Awakening the Classification Power of Faster RCNN (2018, neutral): “We also compare the inﬂuence of size of sampled RoIs during training: [8, 16, 32, 64].”
How Generative Adversarial Networks and Their Variants Work (2017, background): “Notably, Li et al. [69] tries to transform a small object with low resolution into a super resolved large object to make the object more discriminative.”
Deeply Supervised Salient Object Detection with Short Connections (2016, background): “In a variety of computer vision tasks, such as image classification [27], [28], semantic segmentation [29], edge detection [26], [30], object detection [31], [32], and pedestrian detection [33], convolutional neural networks (CNNs) [34] have successfully broken the limits of traditional hand-crafted”