Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors

By Linfeng Zhang, Kaisheng Ma. 2021

263 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement (2023, methodology): “Numerous variants of KD have been proposed, including feature distillation [28, 73], iterative distillation [43, 67], and self-distillation [65, 44, 18, 29].”
Knowledge Distillation from A Stronger Teacher (2022, methodology): “Following [37, 52], we use the same standard training strategies and utilize Cascade Mask R-CNN [2] with ResNeXt-101 backbone as the teacher for two-stage student of Faster R-CNN [23] with ResNet-50 backbone; while for one-stage RetinaNet [24] with ResNet-50 backbone, the RetinaNet with ResNeXt-101…”
Recursive Least Squares for Training and Pruning Convolutional Neural Networks (2022, background): “Knowledge distillation algorithms distill knowledge from the original network to generate a small network [11, 12].”
Channel-wise Knowledge Distillation for Dense Prediction* (2020, methodology): “For object detection, we employ the same teacher and student networks and the training settings as in [42].”
Focal and Global Knowledge Distillation for Detectors (2021, methodology): “And we compare the results with FKD [28], which is another effective and general distillation method.”