Factors in Finetuning Deep Model for Object Detection with Long-Tail Distribution

By Wanli Ouyang, Xiaogang Wang, Cong Zhang, Xiaokang Yang. 2016

213 citations recorded by the source corpus.

Finetuning from a pretrained deep model is found to yield state-of-the-art performance for many vision tasks. This paper investigates many factors that influence the performance in finetuning for object detection. There is a longtailed distribution of sample numbers for classes in object detection. Our analysis and empirical results show that classes with more samples have higher impact on the feature learning. And it is better to make the sample number more uniform across classes. Generic object detection can be considered as multiple equally important tasks. Detection of each class is a task. These classes/tasks have their individuality in discriminative visual appearance representation. Taking this individuality into account, we cluster objects into visually similar class groups and learn deep representations for these groups separately. A hierarchical feature learning scheme is proposed. In this scheme, the knowledge from the group with large number of classes is transferred for learning features in its subgroups. Finetuned on the GoogLeNet model, experimental results show 4.7% absolute mAP improvement of our approach on the ImageNet object detection dataset without increasing much computational cost at the testing stage. Code is available on www.ee.cuhk.edu.hk/ wlouyang/ projects/ImageNetFactors/CVPR16.html.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, methodology): “Data processing methods such as oversampling or undersampling [5,8,65] explicitly simulate the input balance, while cost-sensitive approaches [38,85,18] adjust the update via regularizing the objective.”
Large-Scale Long-Tailed Recognition in an Open World (2019, background): “Arising from long-tail distributions of natural data, it has been extensively studied [41, 61, 3, 30, 62, 34, 29, 49, 6].”
Class-Balanced Loss Based on Effective Number of Samples (2019, background): “It is noteworthy to mention that previous work has also explored other ways of dealing with data imbalance, including transferring the knowledge learned from major classes to minor classes [3, 31, 43, 6, 44] and designing a better training objective via metric learning [17, 47, 45].”
Online Multi-object Tracking Using CNN-Based Single Object Tracker with Spatial-Temporal Attention Mechanism (2017, methodology): “With the development of object detection methods [8, 14, 15, 37, 38], data association [22, 39, 33, 2] has become popular for MOT.”
Overcoming Classifier Imbalance for Long-Tail Object Detection With Balanced Group Softmax (2020, background): “[28] proposes the concept of long-tail object detection, their work focuses on the imbalanced training data distribution on ILSVRC DET dataset [6] without few-shot setting for tail classes like LVIS [14].”
Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation (2020, background): “[29, 31, 35] exploit a single hierarchical tree structure to aid the ine-grained classiication.”
Learning More Universal Representations for Transfer-Learning (2017, methodology): “To diversify the source problem, all the mentioned methods [23], [24], [26], [28] re-label specific categories into nonsemantic ones, i.”
Locally-Transferred Fisher Vectors for Texture Classification (2017, methodology): “When applying the pretrained VGG-VD model to the texture image datasets, it could be intuitive to consider finetuning the model first on the specific dataset [3, 15, 16].”
Parametric Contrastive Learning (2021, methodology): “For the common random resized crop used along with the above two strategies, work of [37] explains that the optimal hyper-parameter for random resized crop is (0.”
DeepID-Net: Object Detection with Deformable Part Based Convolutional Neural Networks (2017, unknown): “Because of the power on learning feature representation, deep models have been widely used for object recognition, detection and other vision tasks [25], [52], [75], [21], [53], [85], [19], [27], [16], [35], [40], [76], [77], [33], [29], [57], [59], [57], [58], [31], [32], [30], [80], [79], [68], [6”
Object Detection With Deep Learning: A Review (2018, background): “found that the samples from different classes follow a longtailed distribution [115], which indicates that different classes with distinct numbers of samples have different degrees of impacts on feature learning.”
Small Sample Learning in Big Data Era (2018, methodology): “Taking the object recognition problem as an example (Ouyang et al., 2016), Rahman et al.”
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets (2020, background): “Object counts in the real world tend to naturally follow a long-tail distribution (Ouyang et al., 2016; Salakhut-dinov et al., 2011; J. Yang et al., 2014).”
Learning to Model the Tail (2017, background): “Even those datasets that are balanced along one dimension still tend to be imbalanced in others [13]; e.”
Range Loss for Deep Face Recognition with Long-Tailed Training Data (2016, background): “[19] investigates several factors that influence the performance of object detection with long tailed distribution of samples.”