Discover the Effective Strategy for Face Recognition Model Compression by Improved Knowledge Distillation

By Rujie Liu, Narishige Abe, Hidetsugu Uchida, Tomoaki Matsunami, Shigefumi Yamada. 2018

5 citations recorded by the source corpus.

For the sake of better accuracy, the face recognition model is becoming larger and larger, which makes them difficult to be deployed on embedded systems. This work proposes an effective model compression method using knowledge distillation, where a fast student model is trained under the guidance of a complex teacher model. Firstly, different loss combinations and network architectures are analyzed through comprehensive experiments to find the most effective approach. To augment the performance, the feature layer is further normalized to make the optimization objective consistent with cosine similarity metric. Moreover, a teacher weighting strategy is proposed to address the issue when teacher provides wrong guidance. Experimental results show that the student model built by our approach can surpass the teacher model while achieving 3× acceleration.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Knowledge Distillation: A Survey (2020, methodology): “…extended to other visual recognition applications, including face recognition (Luo et al. 2016; Kong et al. 2019; Yan et al. 2019; Ge et al. 2018; Wang et al. 2018b, 2019c; Duong et al. 2019; Wu et al. 2020; Wang et al. 2017; Zhang et al. 2020b; Wang et al. 2020b), image/video segmentation (He…”