General Instance Distillation for Object Detection

By Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Sihan Liu, Erjin Zhou. 2021

242 citations recorded by the source corpus.

In recent years, knowledge distillation has been proved to be an effective solution for model compression. This approach can make lightweight student models acquire the knowledge extracted from cumbersome teacher models. However, previous distillation methods of detection have weak generalization for different detection frameworks and rely heavily on ground truth (GT), ignoring the valuable relation information between instances. Thus, we propose a novel distillation method for detection tasks based on discriminative instances without considering the positive or negative distinguished by GT, which is called general instance distillation (GID). Our approach contains a general instance selection module (GISM) to make full use of feature-based, relation-based and response-based knowledge for distillation. Extensive results demonstrate that the student model achieves significant AP improvement and even outperforms the teacher in various detection frame-works. Specifically, RetinaNet with ResNet-50 achieves 39.1% in mAP with GID on COCO dataset, which surpasses the baseline 36.2% by 2.9%, and even better than the ResNet-101 based teacher model with 38.1% AP.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuan Gong
2. Chun Yuan
3. Mang Wang
4. Zhe Li
5. Xiaohu Jiang
6. Zhendong Yang
7. Hangjie Yuan
8. Danpei Zhao

In Computer Vision and Pattern Recognition

1. Yuan Gong
2. Chun Yuan
3. Mang Wang
4. Zhe Li
5. Xiaohu Jiang
6. Zhendong Yang
7. Hangjie Yuan
8. Danpei Zhao

Citations in the corpus

• Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation (2022, background): “Widely applied in image classification tasks in previous researches, knowledge distillation is now used in object detection tasks frequently [8].”
• Focal and Global Knowledge Distillation for Detectors (2021, methodology): “We compare with other two knowledge distillation methods [5, 28] for object detection.”